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Abstract 


The  effect  of  indirect  vision  systems  on  target  detection  and  recognition  is  of  interest  to 
designers  of  future  combat  vehicles.  In  this  report,  a  model  of  target  detection  and  mental 
workload  is  described  for  an  indirect  vision  system  with  a  micro-state  task  time  line 
analysis  of  attention  workload.  The  model  is  based  on  the  results  of  a  field  study  in  which 
participants  detected  and  identified  pop-up  targets  on  an  outdoor  range  from  a  stationary 
and  a  moving  vehicle  while  using  a  head-mounted  display  (HMD),  with  and  without  sound 
localization  and  with  direct  viewing  as  a  control.  A  head-slaved  camera  mounted  on  top  of 
the  vehicle  provided  the  image  to  the  HMD.  As  a  result  of  the  study,  a  target  detection  and 
identification  model  is  derived  for  the  different  treatments  from  the  data  via  regression 
analysis.  The  effect  of  indirect  vision  on  mental  workload  is  determined  from  the  subjective 
ratings  of  perceived  task  loading  that  were  reported  in  the  field  study.  Along  with  the 
perceived  workload,  the  study  participants  rated  the  mental  measures  of  task-allocated 
attention,  situational  awareness,  and  motion  sickness.  Because  of  collinearity,  the  perceived 
workload  is  regressed  on  the  factorial  components  of  a  cognitive  loading  space  derived  from 
a  factorial  analysis  of  the  mental  measures.  Following  rotation  to  a  “skills-rules-knowledge” 
cognitive  processing  space  derived  from  the  clustering  of  the  measures,  the  perceived 
workload  is  shown  to  be  a  function  of  the  skill-  and  rule-based  components.  On  this  basis,  a 
micro-state  time  line  model  is  proposed  for  the  task  information  processing.  In  addition  to 
the  attention  allocated  to  the  targeting  task,  the  model  includes  attention  to  cognitive  maps 
for  target  orienting  by  the  audio  cues  and  the  monitoring  of  the  internal  somatic  state  for 
motion  sickness.  Further,  the  model  includes  switching  between  tasks  (detection  to 
identification)  and  the  initial  orienting  and  subsequent  suppression  responses  to  audio  cues 
and  body  movements  as  intrusive  stimuli.  The  task-loading  measures  generated  by  the 
model  are  compared  to  the  reported  values  with  significant  results.  Finally,  the  model 
results  are  used  to  construct  information  flow  networks  for  task  analysis  workload 
simulation  studies.  The  model  is  intended  for  future  work  in  predicting  fatigue  and 
performance  errors  that  are  induced  by  mental  workload  during  indirect  vision  targeting. 
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MODELING  WORKLOAD  FOR  TARGET  DETECTION  FROM  A  MOVING  VEHICLE 
WITH  A  HEAD-MOUNTED  DISPLAY  AND  SOUND  LOCALIZATION 


1.  Introduction 


In  this  section  of  the  report,  we  describe  the  background  rationale  for  the  research  and  discuss 
factors  to  be  considered  in  the  modeling  effort.  We  comment  on  the  effects  of  camera  and 
display  selection  on  performance  as  well  as  the  benefits  of  sound  localization  for  target 
detection.  We  suggest  criteria  for  optimal  crew  performance,  including  workload  and  situational 
awareness  (SA),  and  we  discuss  the  potential  effects  of  motion  sickness.  Finally,  we  comment  on 
the  field  study  and  experimental  data  used  in  the  modeling. 

1.1  Background 

The  Army  needs  combat  vehicles  that  are  smaller,  lighter,  more  lethal,  survivable,  and  more 
mobile  to  support  a  rapidly  deployable  force.  Combined  with  the  need  to  assimilate  and 
distribute  more  information  to,  from,  and  within  the  vehicle  as  the  Army  moves  toward  a  digital 
battlefield,  there  is  the  need  for  an  increase  in  vehicular  and  command,  control,  communications, 
computers,  and  intelligence  systems  integration  and  performance.  Consequently,  the  Army  will 
need  sophisticated,  highly  integrated  crew  stations  for  these  future  combat  vehicles.  In  support  of 
this  effort,  the  Tank- Automotive  Research,  Development,  and  Engineering  Center  (TARDEC)  is 
developing  the  TARDEC  crew  integration  and  automation  test  bed  (CAT)  advanced  technology 
demonstrator  (ATD).  The  purpose  of  the  CAT  ATD  is  to  demonstrate  crew  interfaces, 
automation,  and  integration  technologies  required  to  operate  and  support  future  combat  vehicles. 
The  results  can  dramatically  increase  the  operational  effectiveness  and  capabilities  with  fewer 
crew  members,  thereby  contributing  to  smaller  and  lighter  weapons  systems. 

1.2  Indirect  Vision  Systems 

To  satisfy  the  Army  requirements  for  reduced  gross  weight  and  lower  silhouette,  as  well  as  the 
need  for  increased  crew  protection  against  ballistic  and  directed  energy  threats,  designers  of 
future  armored  combat  vehicles  will  place  the  crew  stations  deep  within  the  hull  of  the  vehicle. 
For  protection  against  direct  and  indirect  fire  as  well  as  chemical  and  biological  agents,  the  crews 
will  operate  with  their  hatches  closed  and  sealed.  High  intensity  combat  lasers  that  can  penetrate 
direct  vision  blocks  may  force  the  crew  to  operate  on  the  battlefield  in  a  “buttoned  up”  mode, 
with  indirect  vision  systems  for  driving  and  target  search  and  engagement.  The  conventional 
optics,  which  consist  of  periscopic  vision  blocks  and  optical  sights,  will  be  replaced  by  electronic 
displays  at  each  crew  station  and  by  externally  mounted  camera  arrays  on  the  vehicle.  These 
vision  systems  will  show  computerized  digital  images  that  are  electronically  acquired  by  the 
camera  arrays.  The  crew  member  will  see  a  selected  portion  of  the  computerized  display  buffer 
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that  depends  on  his  role  and  viewing  direction.  No  doubt,  by  the  incorporation  of  virtual  reality 
display  and  camera  components,  future  vision  systems  will  appear  to  have  a  “see-through  armor” 
capability  with  a  seemingly  direct  view  of  the  external  scene  through  the  armored  hull  of  the 
vehicle. 

1.3  Camera  Selection 

Before  indirect  vision  systems  can  be  considered  for  future  vehicle  designs,  combat  and  materiel 
developers  will  need  to  know  the  potential  impact  upon  the  crew’s  combat  performance.  During 
night  operations,  replacing  the  vision  blocks  with  infrared  thermal  viewers  improves  the  crew 
performance  by  enhancing  visibility  at  low  light  levels.  In  daylight  conditions,  however,  the  use 
of  indirect  vision  may  cause  a  reduction  in  visual  performance  and  SA.  This  is  because  of  the 
decrease  in  visual  resolution  and  field  of  view  (FOV)  of  the  current  sensors  and  displays  as 
compared  to  vision  of  the  human  eye  through  vision  blocks.  This  reduction  in  visual 
performance  may  reduce  overall  combat  performance.  Further,  the  choice  of  camera 
configuration  and  placement  on  the  vehicle  can  influence  performance.  For  example,  a  single 
telescopic  camera  used  in  target  search  and  detection  may  not  have  the  performance  of  a  multiple 
array  of  such  cameras  placed  about  the  vehicle  and  electronically  integrated  to  give  a  more 
naturally  panoramic  view. 

1.4  Visual  Display  Selection 

The  visual  display  selection  will  influence  the  vehicle  design.  The  design  may  use  a  set  of  panel- 
mounted  displays,  either  cathode  ray  tube  (CRT)  or  flat  panel  liquid  crystal  displays  (LCD), 
which  are  fixed  in  a  panoramic  arrangement  about  the  crew  member’s  station.  Another  option  is 
the  use  of  a  miniature  head-mounted  display  (HMD)  attached  to  the  crew  member’s  helmet.  The 
display  scene  of  the  HMD  can  be  slaved  to  head  movements  with  a  head  tracker  and  for  that 
reason,  may  appear  more  natural  but  with  a  limited  FOV.  Compared  to  the  CRT  and  LCD  panel 
displays,  the  use  of  the  HMD  significantly  reduces  the  size,  weight,  and  power  requirements  for 
the  crew  station.  However,  the  miniature  displays  that  are  currently  available  cannot  match  the 
brightness  and  resolution  of  the  larger  panel  systems  and  may  result  in  degraded  crew 
performance.  The  display  selection  will  also  have  an  impact  on  the  crew  size  needed  to  operate 
the  armored  vehicles  of  the  future.  We  can  expect  in  the  future  to  have  two-  or  three-person 
crews  available.  The  form  of  computerized  aiding  used  with  the  crew  member’s  electronic 
associate  for  the  armored  crew  station  is  influenced  by  the  display  design.  A  panoramic  design  of 
panel  displays  for  a  two-person  crew  seated  together  may  facilitate  team  interaction  and 
performance.  In  contrast,  the  use  of  HMDs  may  tend  to  isolate  the  crew  members  while  requiring 
increased  electronic  communication  between  them. 

1.5  Sound  Localization 

One  problem  of  interest  is  the  effectiveness  of  an  HMD  in  target  detection  and  identification 
with  a  head-controlled  telescopic  sight.  A  problem  for  target  detection  and  identification  with 
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indirect  vision  systems  is  the  inherently  low  visual  acuity  and  limited  FOV  provided  by  the 
current  display  technology.  On  the  modem  digital  battlefield,  the  crew  may  have  advanced 
knowledge  of  the  target’s  location.  In  this  case,  auditory  directional  cueing  by  sound  localization 
should  facilitate  visual  target  detection  by  orienting  the  gunner  toward  the  target.  The  choice  of 
auditory  cueing  for  a  visual  task  reduces  cognitive  interference.  This  is  shown  by  the  literature 
about  interference  for  performance  of  concurrent  tasks  with  directed  attention,  time  sharing,  and 
workload  (Wickens,  1992).  Cueing  can  act  as  a  secondary  task  supporting  or  interfering  with  the 
primary  task,  depending  on  the  display  format  design.  Wickens  (1992)  reports  poor  time  sharing 
for  two  visual  displays  so  spatially  separated  that  both  cannot  be  accessed  by  foveal  vision 
simultaneously.  However,  displays  of  different  modalities  such  as  visual  and  auditory  can  have 
reduced  interference  with  task  performance. 

1.6  Optimal  Crew  Performance 

Another  factor  in  the  display  design  is  the  need  to  maintain  S  A  and  a  mental  model  of  the  task. 
As  noted  by  Endsley  (1993b),  SA  is  a  precursor  to  optimal  performance  since  a  loss  in  awareness 
impacts  decision  making  and  leads  to  a  risk  of  performance  error.  However,  human  operators  can 
perform  well  with  less-than-optimal  display  systems  by  increasing  their  efforts  to  meet  the  more 
demanding  workload.  The  resulting  increase  in  flow  of  information  and  tasks  may  result  in  a  loss 
of  SA;  this  is  because  the  ability  of  humans  to  process  information  is  innately  limited.  For  this 
reason,  besides  demonstrating  improved  performance,  a  further  criterion  is  that  the  display 
system  should  not  generate  excessive  workload  nor  decrease  SA. 

1.7  Motion  Sickness 

Another  factor  influencing  crew  performance  is  the  possibility  of  motion  sickness,  which  can 
occur  in  an  enclosed  cab  area  with  spatial  disorientation.  As  noted  by  Yardley  (1992),  motion 
sickness  is  provoked  by  sensory  conflict  between  the  visual  and  sensorimotor  activities  that 
involve  the  vestibular  system  through  head  movements.  Associated  is  a  constellation  of  mainly 
autonomic  symptoms  such  as  pallor,  drowsiness,  salivation,  sweating,  nausea,  and  finally, 
vomiting  in  the  more  severe  cases  of  motion  sickness.  Although  some  individuals  may 
eventually  adapt  to  situations  that  initially  provoke  sickness  (Yardley,  1992;  Baltzley,  Kennedy, 
Berbaum,  Lilienthal,  &  Gower,  1989),  others  do  not,  and  the  occurrences  may  be  severe  enough 
to  arrest  task  performance  until  the  symptoms  subside. 

1.8  Experimental  Field  Study 

Before  indirect  vision  systems  can  be  designed  for  future  military  vehicles,  materiel  developers 
will  need  to  know  the  potential  impact  of  design  parameters  on  combat  performance.  In  support 
of  this  effort.  The  Human  Research  and  Engineering  Directorate  of  the  U.S.  Army  Research 
Laboratory  (ARL)  is  providing  human  factors  expertise  in  determining  the  effect  of  these  new 
crew  station  technologies  with  a  continuing  series  of  studies  and  investigations.  Because  of  this 
need,  TARDEC  asked  ARL  to  investigate  the  effects  of  indirect  vision  upon  target  search  and 
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detection.  In  this  study,  ARL  conducted  an  experiment  to  determine  the  target  detection  and 
identification  performance  of  vehicle  crews  using  computerized  indirect  vision  with  HMDs  and 
optical  systems.  Further,  we  investigated  the  effects  of  indirect  vision  on  the  mental  workload 
measures  of  attention  allocation  and  perceived  workload,  SA,  and  induced  motion  sickness.  The 
results  were  compared  to  direct  vision  search  as  representative  of  a  “see-through  armor”  vision 
system.  The  experimental  methodology  and  statistical  results  are  reported  in  a  separate  report 
(Smyth,  2002b).  This  report  describes  the  modeling  development  for  task  performance  and 
mental  workload  that  was  derived  from  the  results  of  the  field  experiment.  The  next  section 
summarizes  the  method  and  results  of  the  study  as  background  to  the  modeling  effort. 


2.  Objective 


A  model  of  target  search  and  mental  workload  is  derived  for  an  indirect  vision  system.  As  a 
result  of  the  field  study,  a  task  performance  model  is  derived  for  target  detection  and 
identification  as  a  function  of  the  experimental  treatments.  The  effect  of  indirect  vision  on 
perceived  mental  workload  is  modeled  with  subjective  measures  of  task-allocated  attention,  SA, 
and  motion  sickness  that  resulted  from  the  field  study.  Following  clustering  relations  among  the 
measures,  the  perceived  workload  is  regressed  onto  a  skill-rule-knowledge  (SRK)  cognitive 
attention  space  derived  for  information  processing.  A  micro-state  activity  model  is  proposed  for 
the  targeting  task,  and  the  perceived  workload  and  task  attention  measures  are  related  to  the  time 
line  activities  of  the  model  processors.  The  model  results  are  used  in  task  analysis  workload 
simulation  of  indirect  vision  target  detection  and  identification. 


3.  Field  Experiment 


Described  in  this  section  as  background  information  are  the  methodology  used  in  the  field  study, 
task  analyses  of  the  detection  and  identification  tasks,  mental  model  and  production  rules  used  in 
the  tasks,  and  a  summary  of  the  statistical  results. 

3.1  Methodology 

In  a  field  study,  eight  male  participants  detected  and  identified  “pop-up”  targets  on  an  outdoor 
range  from  a  stationary  and  a  moving  vehicle  while  using  an  HMD,  with  and  without  sound 
localization  and  with  direct  viewing  as  a  control  (Smyth,  2002b).  The  study  was  a  repeated 
measures  experiment  with  all  participants  experiencing  all  conditions  in  a  counterbalanced 
manner.  A  head-slaved  telescopic  camera  mounted  on  top  of  the  vehicle  provided  the  image  to 
the  HMD.  With  sound  localization,  the  audio  tones  (intermittent  at  0.5-second  intervals),  which 
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were  controlled  by  an  on-board  computer,  appeared  to  originate  from  the  location  of  the  target. 
The  targets  were  raised  for  display  and  lowered  one  at  a  time  via  computer  control  by  the 
personnel  supporting  the  experiment.  For  each  trial  mn,  the  experimenter  raised  a  paddle  to  alert 
facility  control  to  start  the  trial  and  then  again  following  target  detection  and  target  identification. 
The  support  personnel  recorded  the  detection  and  identification  times  by  activating  a  computer 
key.  Following  detection,  the  participant  used  binoculars  or  zoomed  the  telescopic  camera  to 
identify  the  orientation  of  a  Landolt  ring  on  the  face  of  the  target  board.  Following  identification, 
the  target  was  lowered  in  preparation  for  the  next  trial.  At  the  end  of  each  counterbalancing 
block  of  trial  runs,  the  participant  rated  questionnaires  on  measures  of  mental  workload, 
including  perceived  workload,  task  attention,  SA,  and  motion  sickness. 

3.2  Task  Analysis 

Figures  1  and  2  are  flow  diagrams  of  the  task  analyses  for  the  target  detection  and  identification 
in  terms  of  the  observable  body  movements  and  responses  from  the  start  of  the  trial  run  to  the 
vocal  response  to  the  experimenter. 


Figure  1.  Task  Analysis  for  Target  Detection. 


Figure  2.  Task  Analysis  for  Target  Identification. 


3.2.1  Detection 

As  seen  from  Figure  1,  at  the  start  of  the  trial  run,  the  participant  looks  for  the  target  with  a 
visual  glimpse  and  tells  the  experimenter  if  one  is  detected.  In  this  process,  he  automatically 
segments  the  glimpsed  image  into  target-sized  shapes  and  compares  them  to  a  mental  template 
that  he  has  memorized.  If  no  target  is  located,  the  participant  continually  indexes  his  eyes  along 
the  experimental  facility  for  more  visual  glimpses.  While  doing  this,  he  compares  the  scene  to  a 
mental  model  that  he  has  of  the  terrain  and  perhaps  to  the  locations  of  targets  that  he  expects  to 
find  on  the  facility.  When  the  participant’s  foveal  vision  is  obscured  by  the  side  of  the  viewing 
device,  he  rotates  his  head  sideways  and  realigns  his  eyes  to  repeat  the  viewing  process.  If  he 
visually  transverses  the  entire  front  of  the  target  range  without  target  detection,  he  will  reverse 
his  head  motion  to  track  back  over  the  facility.  Again,  the  participant  has  to  maintain  a  mental 
map  of  the  terrain  and  his  location  on  the  facility. 

3.2.2  Identification 

Following  detection,  the  participant  places  the  binoculars  to  his  eyes  and  adjusts  the 
magnification  power  while  looking  at  the  target.  He  or  she  continues  to  index  his  head 
orientation  to  keep  the  target  centered  while  he  adjusts  the  magnification.  Similar  comments 
apply  to  viewing  with  the  HMD.  In  this  process,  the  participant  maintains  a  mental  image  of  the 
target  location  in  the  surrounding  scene  to  guide  his  vision  during  the  adjustments.  Furthermore, 
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he  has  an  expectancy  about  the  size  and  location  of  the  Landolt  ring  on  the  target  board,  as  well 
as  the  possible  gap  orientations.  Upon  identification,  he  tells  the  experimenter  the  gap  orientation 
to  complete  the  trial  run. 

3.2.3  Intrusive  Tasks 

Although  the  detection  and  identification  tasks  are  serial  in  nature.  Figure  3  contains  flow 
diagrams  for  two  tasks  that  are  intrusive  to  the  process:  the  responses  to  the  sound  cue  and  scene 
movement.  A  sound  cue  during  detection  will  cause  the  participant  to  align  his  viewing  direction 
with  that  of  the  apparent  source.  However,  sound  cueing  will  not  aid  identification  once  he  has 
sighted  the  target,  and  he  may  decide  to  ignore  the  cue.  In  this  case,  the  sound  cue  may  impose 
an  increased  mental  load  because  of  the  need  to  evaluate  the  intrusion.  A  change  in  the  scene 
induced  by  vehicular  motion  should  cause  the  participant  to  realign  his  viewing  direction.  In  this 
case,  he  is  revising  and  maintaining  a  mental  model  of  the  terrain  and  his  location  and  orientation 
relative  to  the  facility. 


(a).  Sound  cue  subtask. 


(b).  Body  movement  subtask. 


Figure  3.  Intrusive  Sub-tasks  of  Sound  Cues  and  Body  Movement. 

3.2.4  Function  and  Task  Allocation 

Table  1  lists  the  allocation  of  functions  and  tasks  for  the  target  detection  and  identification.  The 
table  lists  the  human  information  processors  and  modalities,  the  operator  interfaces,  and  the 
mental  model  maintenance  that  is  involved  by  sub-task.  The  human  processors  include  the 
perceptual  with  visual  and  acoustic  modalities;  cognitive  with  skill-based,  rule-based,  and 
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Table  1.  Allocation  of  Functions  and  Tasks 
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supervisory  levels;  and  motor  with  ocular,  vocal,  and  manual  modalities.  The  operator  interfaces 
include  the  event  that  elicits  the  sub-task,  the  interface  display  pertinent  to  the  task,  the  feedback 
from  the  operator,  and  the  ensuing  action.  Finally,  the  table  lists  the  impact  of  the  task 
performance  on  the  maintenance  of  the  mental  model  at  both  the  task  level  and  that  of  the  global 
SA. 


3.3  Task  Information  Processing 

In  this  section  of  the  report,  we  review  a  descriptive  model  for  human  information  processing 
appropriate  for  the  target  detection  and  identification  tasks.  In  the  following  section,  a  mental 
model  and  production  rules  are  discussed  for  the  tasks. 

3.3.1  Task-Directed  Behavior 

Task-directed  behavior  may  be  thought  of  as  composed  of  hierarchical  components:  a  sequential 
activity  for  task  performance  and  an  accompanying  assessment  activity  used  to  maintain  a 
mental  model  of  the  process.  The  mental  model  (Norman,  1988)  contains  a  knowledge-based 
understanding  of  the  task  and  SA.  The  task  sequence  consists  of  skill-level  behavior,  which  is 
directed  by  a  rule-based  script  type  schema  (Brookhuis,  De  Vries,  &  De  Waard,  1991)  developed 
for  the  particular  problem  from  the  mental  model.  An  awareness  of  the  surrounding  problem 
structure  is  maintained  to  compose  the  schemas  needed  as  new  problems  arise.  As  noted  by 
Endsley  (1993a,  1993b,  1996),  this  awareness  can  occur  in  the  individual  at  several  levels.  These 
levels  are  (1)  awareness  of  the  problem,  (2)  knowledge  of  the  problem  class  derived  from  pattern 
matching  and  the  state  of  the  elements  comprising  the  problem  structure  for  that  class,  and  (3)  an 
assessment  of  the  consequences  and  a  mapping  of  the  problem  and  the  elements  to  an 
appropriate  solution  scheme.  Thus,  SA  is  determined  by  the  knowledge  of  the  operator  and  his 
understanding  of  goals  and  tactics  (Selcon,  Taylor,  &  Koritsas,  1991). 

The  maintenance  of  a  mental  model  and  the  accompanying  S  A  is  necessary  for  the  continual 
planning  of  problem  involvement  and  itself  constitutes  a  form  of  secondary  task  with  its  own 
unique  workload  requirements.  For  this  reason,  problem  solving  is  composed  of  dual  tasks, 
which,  while  competing  for  attentional  resources,  consist  of  a  sequential  activity  focused  on  a 
particular  problem  and  an  accompanying  maintenance  of  SA  of  the  problem  environment. 
Performance  is  improved  when  the  task  demand  is  matched  by  the  supply  of  knowledge  and 
central  processing  resources  in  the  human.  Central  processing  may  be  conceived  as  drawing 
upon  the  cognitive  resources  according  to  the  level  of  processing  involved;  that  is,  skill-,  rule-,  or 
knowledge-based  behavior  (Rasmussen,  1983, 1986, 1993).  Here,  skill-based  behavior  provides 
the  task  performance,  rule-based  behavior  provides  the  governing  schema  for  skill  control,  and 
knowledge-based  provides  the  schema  formation  for  the  next  task  problem.  Workload  and 
awareness  are  related  since  attentional  resources  are  used  to  acquire  and  maintain  awareness.  In 
turn,  awareness  of  the  task  situation  is  needed  for  effective  decision  making  and  implementation. 
As  with  any  task,  awareness  of  the  situation  is  limited  by  the  attention  capacity  of  the  human 
(Wickens,  1992).  The  implication  is  that  models  of  human  information  processing  should  include 
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a  component  in  memory  which  is  specialized  for  SA  and  a  mental  model  of  the  task  problem 
process  (Gordon,  1997).  Further,  the  model  should  include  the  attentional  demands  of  complex 
decision  making  and  problem  solving  which  are  used  to  maintain  SA. 

3.3.2  Mental  Model 

Table  2  lists  a  mental  model  for  the  detection  and  identification  task  in  terms  of  goals  and 
actions.  In  particular,  the  table  lists  the  goals,  operators,  methods,  and  selection  rules  that  could 
be  used  in  a  goals,  operators,  methods,  and  selection  rules  (GOMS)  model  of  the  task  (Card, 
Moran,  &  Newell,  1983;  Kieras,  1988).  In  this  formulation,  the  human  gunner  has  goals  that  are 
achieved  by  specific  methods  and  selection  rules.  A  method  is  defined  as  a  script  sequence  of 
steps  that  are  performed  by  the  perceptual,  cognitive,  and  motor  operators.  There  are  usually 
several  methods  that  need  to  be  performed  to  accomplish  a  goal.  Selection  rules  are  used  to 
specify  the  conditions  that  must  be  satisfied  for  a  method  to  be  executed  (Wickens,  Gordon,  & 
Lui,  1998). 

Associated  with  the  selection  rules  are  parameters  and  features  used  in  the  decision  making 
process.  Table  3  lists  a  reference  frame  that  would  be  held  in  memory  for  the  parameters  and 
features  used  with  the  selection  rules  of  the  detection  task.  The  reference  values  are  the  values  of 
the  features  used  for  the  selection  criteria.  In  the  gunner’s  model,  the  mental  model  reference 
frame  is  part  of  working  memory  with  the  features  values  being  filled  by  the  perceptual 
processors  as  the  task  is  executed.  The  reference  values  are  established  during  training.  The  rule- 
based  processor  calls  the  selection  rules  according  to  the  task  script  as  each  rule  is  processed  in 
turn.  Associated  with  each  selection  rule  is  the  set  of  features  and  reference  values  used  in 
processing. 

3.3.3  Task  Production  Rules 

The  selection  rules  for  the  detection  and  identification  task  are  in  the  form  of  conditional 
production  rules  that  are  processed  by  the  rule-based  processor.  As  listed  in  Table  4,  the 
production  rules  are  in  an  “if-then”  format  in  which  actions  are  activated  (i.e.,  “fired”)  when  the 
conditions  of  the  parameters  are  satisfied.  The  rules  are  written  in  a  C-language  format  with  the 
function  calls  referring  to  perceptual  and  autonomous  skill-based  processor-driven  motor 
activities. 

3.3.4  Intrusive  Task  Production  Rules 

Table  5  lists  production  rules  for  the  intrinsic  tasks.  By  itself,  body  movement  relative  to  the 
visual  scene  causes  the  participant  to  momentarily  evaluate  his  mental  model  of  his  position  in 
the  scene.  As  shown  in  Figure  3,  body  movement  imposes  an  additional  sub-task  in  the  SA 
domain  that  is  time  shared  with  the  procedures  of  the  structured  task.  However,  vehicular  motion 
may  lead  to  inactivity  because  of  motion  sickness.  In  this  process,  motion  sickness  forces  a 
monitoring  of  discomfort  awareness  as  a  secondary  task  competing  for  attentional  resources.  In 
some  cases,  motion  sickness  dominates  the  responses,  causing  the  participant  to  momentarily 
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discontinue  the  primary  task.  Although  the  effect  of  total  severity  may  be  interpreted  as  the 
imposition  of  additional  stressors  that  act  as  performance-shaping  functions  for  the  allocation  of 
attention  to  the  monitoring  sub-task,  the  corresponding  symptoms  may  have  masking  effects  on 
the  separate  attention-processing  channels.  In  this  sense,  the  symptoms  of  motion  sickness  are 
degrading  processing  channels.  For  example,  the  oculomotor  symptom  may  interfere  with  the 
visual  processing  channel,  while  the  disorientation  symptom  may  interfere  with  the  cognitive 
channel  through  the  impact  on  SA,  and  finally,  the  nausea  symptom  may  appear  to  be  an  added 
attention  loading  on  an  activated  somato-sensory  channel  that  is  otherwise  dormant. 

Table  2.  Mental  Model  of  Target  Detection  and  Identification  Tasks 


Goal  -  Find  target  on  the  experimental  facility  and  identify  as  quickly  as  possible. 

Operators  -  Visually  search  and  identify  by  direct  vision  or  with  head-controlled  telescopic  camera  using  eye  and 
head  movements,  manual  manipulation  of  binoculars  or  telescopic  zoom  control,  and  voice  responses. 

Method  -  Following  verbal  directive  from  experimenter,  start  search  over  front  of  experimental  facility  with  direct 
vision  or  head-controlled  telescopic  camera.  Inform  experimenter  when  target  is  located.  Depending  on  the  search 
method,  raise  binoculars  to  face  or  increase  telescopic  zoom  and  focus  on  target  to  determine  orientation  of  target 
marker,  and  inform  experimenter. 

Selection  rules  -  The  rules  used  by  the  participant  in  executing  the  method: 

(1)  The  experimenter  informs  the  participant  when  the  trial  is  to  start. 

(2)  The  target  is  a  human  silhouette  on  an  experimental  facility  in  the  right  front  sector  from  50  meters  to  250 
meters. 

(3)  Direct  vision  or  an  HMD  may  be  used  in  the  search  and  identification  processes;  the  vehicle  from  which  the 
search  is  made  may  be  stationary  or  moving  along  side  the  range;  and  a  localization  sound  apparently  emitting  from 
the  direction  of  the  target  may  be  presented  throughout  the  search  period. 

(4)  The  search  is  conducted  in  a  sequence  of  visual  glimpses,  starting  at  the  front  and  proceeding  to  the  right  until 
the  sector  side  is  reached. 

(6)  Once  the  target  is  sighted,  a  verbal  response  of  “target  sighted”  should  be  made  to  the  experimenter,  and 
identification  should  commence. 

(7)  In  the  identification  procedure,  the  visual  field  of  a  pair  of  binoculars  or  of  the  HMD  should  be  centered  on  the 
target  and  the  field  zoomed  on  the  target  by  manual  adjustment  until  the  identification  marker  can  be  seen. 

(8)  The  identification  marker  is  a  Landolt  C-ring  on  the  face  of  the  target  with  the  ring  gap  oriented  up,  down, 
right,  or  left. 

(9)  Upon  identification,  a  verbal  response  “up,”  “down,”  “right,”  or  “left”  describing  the  ring  orientation  should  be 
made  to  the  experimenter. 

(10)  The  trial  ends  with  the  identification  response  or  when  the  experimenter  so  commands. _ 
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Table  3.  Frame  of  Reference 


Parameter 

Features 

Reference  Values 

Trial  status 

start 

START  TRIAL 

target  search 

SEARCH 

detection  made 

TGT  DETECTED 

detection  response 

1st  VOICE  RESPONSE 

target  image  magnification 

ZOOM  IMAGE 

identification  made 

TGT  IDENTIFY 

identity  response 

2nd  VOICE  RESPONSE 

end 

END  TRIAL 

Vehicle  status 

stationary 

STATIONARY 

moving 

MOVING 

Search  method 

direct  vision 

DIRECT  VIEW 

HMD 

HMD 

Identification  method 

binoculars 

BINOCULARS 

telescopic  zoom 

TELESCOPIC 

Cueing  method 

none 

NO  CUE 

sound  localization 

3D  AUDIO 

target  status 

presented 

TGT  PRESENT 

none 

NO  TGT 

Target 

shape 

HUMAN_SILHOUETTE 

Target  location 

place 

viewing  sector 

Detection  response 

verbal 

“target  detected” 

Marker 

shape 

LANDOLT  C-RING 

Marker  feature 

gap  orientation 

UP,  DOWN,  LEFT,  RIGHT 

Controller  status 

visual  field  zoom 

ZOOM_LEVEL 

Identity  response 

verbal 

“Oriented  up  (down)  (left)  (right)” 

Vehicle  position 

relative 

viewing  sector 

Vehicle  orientation 

relative 

viewing  sector 

Vehicle  speed 

optical 

flow  field 

Scene  texture 

scene  blob  shape,  size,  color 

terrain  features 

Velocity  flow  field 

displacement  of  terrain  texture 

FLOW  SEPARATION 

Body  awareness 

motion  sickness 

NONE,  SICK,  INCAPACITATED 

Eye  orientation 

relative 

head 

Head  orientation 

relative 

vehicle 

Audio  cue  status 

on 

CUE  ON 

off 

CUE  OFF 

Audio  cue  response 

respond 

ORIENT 

ignore 

SUPPRESS 

Trial  start 

verbal  command 

“target  up” 

Trial  end 

identity  response 

verbal 

verbal  command 

“end  trial” 

12 


Table  4.  Production  Rules  for  Target  Detection  and  Identification  Tasks 


//  detection: 

inactivity:  //  wait  for  trial  to  start 

if(  hear(“target  up”))  process_glimpse(); 
else 

goto  inactivity; 

process  glimpse() 

{ 

glimpseQ; 

if(beyond_FOV(Search_method)) 

{  index_head(); 
reset_eyefield(); 
if(beyond  recall(course))  { 
reverse_head(); 
index_head(); 
reset_eyefield(); }  } 
else  index_eyefield(); 
process_glimpse(); 

} 

glimpseQ 

{ 

blob_sceneO; 

if(image_blob  mapsto  recall(target_shape)) 
{voicejell  (“detection”); 
goto  identify;  } 

} 

//  identification:: 
identify: 

if(direct__vision)  place_binoculars(); 
continue_ID: 

if(voice_tell(“stop”))stop(); 

if(on_target  !=  TRUE)  orient_on_target(); 

adjust_zoom(); 

blob_scene(); 

if(image_blob  maps_to  recall(C_ring))  { 
decision(gap_orientation); 
voice_tell  (gap_orientation); 
stop();  } 
else 

goto  continueJOD; _ 
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Table  5.  Production  Rules  for  Intrusive  Tasks 


//  sound  cue  intrusion: 

sound_cue() 

{ 

decision(); 

if(decide  ==  respond) 

{ 

local  ize_on_cue(); 
align_head(); 

} 

else 

{ 

suppress(); 

} 

return();  } 

} 

//  body  movement  intrusion: 

bodyjnovementO 

{ 

if(discomfort_aware()  ==  sickness) 

somatic_response(); 

else 

place_self(  vehicle, range); 

J _ 

Note:  Ail  function  calls  are  at  perceptual  processors  or  autonomous  skill  level. 

3.4  Summary  of  Statistical  Results 

The  results  of  the  field  study  are  that  more  targets  were  detected  with  direct  viewing  than  with 
the  HMD  and  from  the  stationary  position  than  the  moving  vehicle.  Although  more  targets  were 
detected  with  direct  viewing  from  the  stationary  vehicle  without  cueing,  sound  localization 
improved  target  detection  in  all  other  treatments.  Similar  comments  apply  to  identification  from 
the  stationary  position;  however,  fewer  targets  were  identified  from  the  moving  vehicle  with 
sound  localization  than  without.  The  advantages  of  auditory  cueing  for  target  detection  may  have 
been  limited  by  the  choice  of  an  intermittent  tone  for  sound  localization  and  the  restricted  search 
sector  used  in  the  experiment. 

Ratings  of  task  attention  loading  show  that  increased  attention  was  needed  for  the  detection  task 
with  sound  localization,  especially  with  the  HMD.  This  is  also  true  for  the  identification  task. 
Workload  test  batteiy  ratings  show  a  significant  increase  in  perceived  workload  with  the  HMD 
as  compared  to  the  direct  vision.  A  test  of  SA  shows  significant  effects  with  decreasing  trends  in 
perceived  stability  and  familiarity  and  an  increased  need  to  concentrate  when  one  is  using  the 
HMD  in  the  moving  vehicle.  Further,  most  of  the  participants  reported  a  general  discomfort 
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associated  with  motion  sickness  while  in  the  moving  vehicle  with  the  HMD.  There  was  no 
significant  change  in  heart  rate,  with  test  mode  indicating  no  impact  of  metabolic  work  on  their 
performance.  As  a  result  of  the  study,  the  participants  rated  direct  viewing  as  more  useful  than 
the  HMD  and  the  sound  localization  as  being  of  more  use  with  the  HMD  than  with  direct 
viewing. 

3.4.1  Statistical  Values  of  Measures 

For  modeling  workload,  the  statistical  values  for  the  perceived  workload,  attention  allocation, 
SA,  and  motion  sickness  are  of  interest.  Table  6  lists  the  averages  and  standard  deviations  for  the 
statistically  significant  mental  workload  measures  as  a  function  of  the  pertinent  treatments. 

These  measures  are  for  the  questionnaires  on  Task  Loading  Index  (TLX),  motion  sickness,  and 
Situation  Awareness  Rating  Technique  (SART),  described  in  greater  detail  along  with  the 
attention  allocation  questionnaire,  further  in  the  report.  The  maximum  possible  scores  are  listed 
for  reference.  Note  that  the  SART  supply  measures  are  considered  to  be  insignificant  by 
treatments.  Table  7  lists  the  average  scores  for  the  statistically  significant  attention  allocation 
factors  for  the  detection  and  identification  tasks.  The  equivalent  verbal  anchors  are  listed,  along 
with  the  scores  for  reference.  Note  that  the  maximum  possible  allocation  score  on  an  attention 
channel  is  7. 


Table  6.  Statistically  Significant  Mental  Workload  Questionnaire  Results 


Measure 

Average 

SD 

Number 

Total  TLX: 

Direct  viewing 

13.966 

9.958 

32 

HMD  viewing 

28.100 

11.011 

32 

Motion  Sickness  Severity: 

All  other  cases 

4.414 

6.776 

58 

HMD  &  motion 

22.199 

23.342 

16 

SART  measures: 
Cal  Demand: 


Direct  viewing 

8.225 

4.279 

32 

HMD  viewing 

11.538 

4.614 

32 

(b)  Supply.: 

Arousal 

3.362 

1.735 

64 

Capacity 

3.933 

1.853 

64 

Concentration 

4.586 

1.497 

64 

Division 

3.753 

1.811 

64 

( cl  Understanding: 

Direct  viewing 

14.672 

3.924 

32 

HMD  viewing 

12.216 

5.284 

32 

SD  =  standard  deviation 
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Table  7.  Statistically  Significant  Attention  Loading  Factors 


Average 

Configuration  Number  Value3  SD  Verbal  Anchor 


I.  DETECTION  TASK 
(a)  Visual  channel: 


All  treatments 

64 

3.575 

2.190 

“discriminate” 

(b)  Cognitive  channel: 

All  treatments 

64 

3.367 

1.598 

“recognize” 

(c)  Audio  channel: 

Non  cueing  treatments 

32 

0.062 

0.354 

“none” 

Sound  cueing  treatments 

32 

3.293 

1.574 

“orient,”  “verify” 

(d)  Psychomotor  channel: 
Direct  vision 

HMD  vision 

32 

32 

1.788 

3.781 

1.538 

1.519 

“toggle” 

“continuous  ”  “manipulate” 

II.  IDENTIFICATION  TASK 

(a)  Visual  channel: 

All  treatments 

64 

3.973 

1.957 

“inspect” 

(b)  Cognitive  channel: 

All  treatments 

64 

3.667 

1.916 

“recognize” 

(c)  Audio  channel: 

Non  cueing  treatments 

32 

0.000 

0.000 

“none” 

Sound  cueing  treatments 

32 

2.725 

1.683 

“orient,”  “verify” 

(d)  Psychomotor  channel: 

All  treatments 

64 

3.039 

1.880 

“continuous” 

T^ote:  maximum  possible  value  on  all  channels  equals  7. 

4.  Task  Performance  Model 

A  model  is  developed  by  regression  analysis  for  the  time  to  detect,  the  detection  event,  the  time 
to  identify,  the  identification  event,  and  the  correct  identification  from  the  experimental  data. 

4.1  Time  to  Detect 

Given  that  the  target  is  visible,  the  time  to  look  at  the  target  is  the  time  to  shift  the  gaze  direction 
from  the  range  front  to  the  location  of  the  target.  The  search  strategy  is  one  of  shifting  a  visual 
search  lobe  across  the  visual  front  by  a  sequence  of  rapid  eye  movements  integrated  with 
intermittent  head  movements.  The  rate  of  shifting  the  search  lobe  is  determined  by  the  time  for 
the  human  to  (a)  acquire  a  visual  image  for  each  lobe  fixation,  (b)  perceptually  isolate  pertinent 
features  from  the  image,  and  (c)  cognitively  compare  the  features  to  a  physical  template  for  the 
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target.  Given  a  self-paced  search  task,  this  is  an  integrated  activity  of  pipelined  visual, 
perceptual,  and  cognitive  processes. 

In  natural  vision,  the  size  of  the  search  lobe  is  determined  by  the  expected  conditions  for  the 
target  size  and  background;  the  search  rate  is  determined  by  the  angular  size  divided  by  the 
processing  time.  The  search  rate  is  lower  for  the  HMD,  since  the  angular  size  of  the  search  lobe 
is  reduced  by  the  restricted  view  of  the  camera.  Furthermore,  the  processing  time  is  increased  by 
the  need  to  cognitively  relocate  the  viewing  direction  on  the  course  as  the  vehicle  moves 
forward. 

The  sound  cue  narrows  the  search  sector  by  orienting  the  participant  toward  the  target.  In  the 
case  of  direct  viewing,  the  search  lobe  is  naturally  wide,  and  the  synthetic  stereophonic  process 
proved  to  be  perceptually  insensitive  for  the  limited  search  sector.  In  contrast,  the  FOV  of  the 
HMD  is  narrower  than  the  cued  sector.  In  this  case,  while  sound  localization  initially  orients  the 
participant  toward  the  target,  this  is  followed  by  a  refined  visual  search. 

Further,  sound  localization  bypasses  the  visual  processor  and  involves  the  human  acoustic 
processor  for  signal  isolation,  the  cognitive  processor  for  comparing  the  stereophonic  signals, 
and  ocular  and  head  motor  adjustment  to  align  the  signals.  In  practice,  because  of  the  intermittent 
tone  and  limited  search  sector  used  in  this  experiment,  the  sound  localization  may  have  increased 
the  processing  time  by  adding  to  the  visual  task.  The  intermittent  tone  caused  the  participant  to 
momentarily  wait  between  tones  to  make  adjustments,  and  visual  processing  occurred  during  the 
off-duty  cycle. 

For  these  reasons,  the  time  to  detect  (td),  given  a  detection,  is  the  angular  separation  (4>E)  of  the 
target  from  the  participant’s  initial  orientation  divided  by  the  size  of  the  search  lobe  (<t>£), 
multiplied  by  the  inverse  of  the  rate  of  search.  Without  cueing,  the  separation  angle  is  the  angle 
of  the  target  from  the  range  front  (<t>T).  With  cueing,  the  angle  is  for  orientation  (<t>x).  The  search 
lobe  size  is  different  for  direct  viewing  and  the  HMD.  The  inverse  rate  of  search  is  represented 
by  a  natural  vision  constant  (Ko)  multiplied  by  adjustments  for  the  treatments  of  vehicular 
motion  (am),  use  of  the  HMD  (av),  and  considering  the  statistical  results,  by  the  interaction  of 
the  sound  localization  with  the  motion  and  vision  modes  (as).  The  detection  time  is 

td  =  ts+  [K0*am*av*as]*<|)E/<l>I, 

in  which  the  adjustments  are  unity  for  those  cases  when  the  treatment  conditions  do  not  apply.  A 
fixed  time  component,  ts,  has  been  added  to  account  for  the  reaction  times  of  the  participant  to 
the  verbal  command  to  start  the  trial,  the  participant  to  verbally  respond  to  his  sighting,  the 
experimenter  to  the  verbal  response,  and  the  facility  control  to  the  experimenter’s  flag.  Taking 
the  natural  logarithm  of  both  sides,  we  have  the  linear  expression, 

ln(td  -  ts)  =  Po  +  Pm*am  +  Pv*«v  +  Ps*as  +  pT*ln(4>E/<te). 
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in  terms  of  the  effects  coding  (a  =  0  or  1)  for  the  treatments  and  the  corresponding  powers  ((3)  of 
the  natural  exponential  as  regression  coefficients.  Here,  a  coefficient  has  been  included  for  the 
target  angle;  the  viewing  lobe  is  assumed  fixed  in  size.  A  stepwise  linear  regression  fit  of  the 
equation  to  the  detection  times  for  those  targets  detected  is  statistically  significant  (adjusted 
R-square  =  0.399, p  <  .001,  F  =  1 13.292,  df  =  3,  dfe  =  504),  with  significant  coefficients  for  the 
vehicle  motion  (bm  =  0.156,/?  <  .001)  and  viewing  with  the  HMD  (by  =  0.587,/?  <  .001),  and 
the  interaction  with  sound  localization  excluded  by  the  analysis.  The  vehicle  motion  and  HMD 
both  increase  the  time  to  detection.  However,  while  the  regression  coefficient  for  the  target  angle 
is  significant  (bj  =  0.156,/?  <  .001),  the  value  is  much  less  than  unity.  The  detection  times  are 
increased  for  the  targets  near  the  extreme  edge  of  the  search  sector  but  not  for  those  toward  the 
range  front  or  between.  This  implies  that  the  search  sector  (60  degrees)  was  too  narrow  to  force 
different  search  strategies  for  the  different  treatments  except  at  the  extreme  target  angles. 

4.2  Detection  Event 

The  detectability  of  a  viewed  target  is  a  direct  function  of  its  visibility.  For  a  distant  outdoor 
target,  the  visibility  is  a  function  of  its  shape,  apparent  area,  range,  and  brightness  contrast. 
Additional  factors  are  the  viewer’s  visual  acuity  and  adaptation  brightness.  For  outdoor  viewing, 
the  brightness  contrast  is  determined  by  the  sun  angle  relative  to  the  viewing  direction  and  the 
reflectance  of  the  target  and  its  surroundings  (Smyth,  1988).  The  target  range  used  in  this  study 
is  oriented  in  the  east-west  direction,  and  all  viewing  was  toward  the  east.  Experimental  trials 
were  performed  from  10  a.m.  to  2  p.m.  to  ensure  an  overhead  sun. 

Given  that  the  participant  is  looking  at  the  target,  detection  occurs  if  the  visibility  of  the  target 
exceeds  a  threshold  value.  The  effective  area  is  the  true  area  adjusted  by  the  cosine  of  the  target 
angle  to  the  observer.  The  HMD  may  reduce  the  visibility  because  of  the  reduced  resolution  of 
the  display.  Similarly,  vehicle  motion  reduces  the  resolution  by  motion  blurring  of  the  image. 
Sound  localization  may  effectively  increase  the  visibility  by  increasing  the  participant’s  signal 
detection  sensitivity;  however,  the  statistical  results  show  no  such  effect.  Furthermore,  the 
statistics  show  dependency  upon  the  interaction  between  the  HMD  and  vehicle  motion.  For 
daytime  viewing,  we  assume  that  the  adaptive  brightness  is  constant.  For  these  reasons,  the 
visibility  (Vj)  is  the  product  of  adjustments  for  the  vehicle  motion  (am),  use  of  the  HMD  (av), 
and  the  interaction  between  the  two  (amxv),  along  with  the  target  area  (A),  cosine  of  the  target 
angle  (<J>T),  brightness  contrast  (be),  and  visual  acuity  (ac)  of  the  participant,  all  divided  by  the 
square  of  the  target  range  (pT).  That  is, 

VT  =  Ko*am*av*amxv*ac*bc*A*cos((()T)/pT^- 

Taking  the  natural  logarithm  of  both  sides,  we  have  the  linear  expression 

ln(Vj)  =  Po  +  Pm*otm  +  Pv*0Cy  +  Pmxv*«mxv  +  PT*lu[ac*bc*A*cos((j>T)/pT^], 

in  terms  of  the  effects  coding  (a  =  0  or  1)  for  the  treatments  and  the  corresponding  powers  (P)  of 
the  natural  exponential.  Here,  a  coefficient  has  been  included  for  the  target  visibility. 
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Considering  a  discriminant  function  (Dt)  which  is  unity  for  a  successful  detection  and  zero  for  a 
failure  to  detect,  a  stepwise  regression  analysis  of  the  target  detection  is  significant  (adjusted  R- 
square  =  0.366,  p  <  .001,  F  =  148.268,  df  =  3,  dfe  =  762),  with  significant  coefficients  for 
viewing  with  the  HMD  (by  =  -0.343,/?  <  .001)  and  motion  with  the  HMD  (bmxv  =  -0.227, 
p  <  .001),  with  however,  the  vehicle  motion  excluded.  As  expected,  the  HMD  and  motion  with 
the  HMD  both  decrease  the  target  visibility.  However,  while  the  regression  coefficient  for  the 
target  visibility  is  significant  (bj  =  0.149 ,/?  <  .001),  the  value  is  much  less  than  unity;  the 
detection  events  are  decreased  by  the  targets  near  the  far  range  of  the  search  sector  but  not  by 
those  that  are  close  or  between. 

4.3  Time  to  Identify 

Once  the  target  has  been  detected,  the  time  to  identify  is  the  time  that  the  participant  needs  to 
magnify  the  target  image  so  that  the  Landolt  ring  gap  is  visible.  The  time  is  a  function  of  whether 
binoculars  or  electronic  zooming  is  used.  Also,  vehicular  motion  reduces  resolution  by  blurring 
the  image.  However,  since  the  image  has  been  located,  sound  localization  provides  no  assistance 
if  visual  focus  on  the  image  is  maintained.  According  to  the  statistical  results,  the  interaction  of 
the  vehicle  motion  with  the  HMD  has  an  effect.  Another  factor  is  the  target  range  since  a  distant 
target  will  have  to  be  viewed  at  a  higher  magnification  for  the  identification  marker  to  be  seen. 
Furthermore,  the  gap  area  (Ag)  seen  is  a  function  of  the  cosine  of  the  target  angle,  4>T-  Also,  the 
participant’s  visual  acuity  determines  the  magnification  level  needed  to  see  the  marker  image.  In 
this  case,  the  brightness  contrast  is  not  a  factor  since  it  is  fixed  for  the  marker  on  the  target  board. 
The  adjustment  will  take  longer  for  those  targets  that  need  to  be  seen  at  a  higher  magnification. 
The  expression  for  the  identification  time  including  the  response  times,  is 

tid  =  ts  +  Ko*am*av*amxv*[pT^/(ac*Ag*cos((|)T))]. 

Taking  the  natural  logarithm  of  both  sides,  we  have  the  linear  expression 

ln(tid  -  ts)  =  po  +  Pm*am  +  Pv*av  +  Pmxv*amxv  +  pTHn[px2/(ac*Ag*cos(<|)T))], 

in  terms  of  the  effects  coding  (a  =  0  or  1)  for  the  treatments  and  the  corresponding  powers  (P)  of 
the  natural  exponential.  A  stepwise  linear  regression  fit  of  the  equation  to  the  identification  times 
for  targets  identified  is  significant  (adjusted  R-square  =  0.182 ,p  <  .001,  F  =  28.563,  df  =  3, 
dfe  =  368),  with  significant  coefficients  for  the  vehicle  motion  (bm  =  0.1 13,/?  <  .001)  and 
viewing  with  the  HMD  (bv  =  0.109,/?  <  .001),  with  the  interaction  excluded.  As  expected,  the 
vehicular  motion  and  use  of  the  HMD  increase  the  time  to  identify.  As  before,  the  regression 
coefficient  for  the  target  is  significant  (bx  =  0.076,/?  <  .001).  Since  the  value  is  much  less  than 
unity,  the  identification  times  are  increased  by  the  targets  that  are  near  the  far  range  of  the  search 
sector. 
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4.4  Identification  Event 


Given  that  the  participant  is  looking  at  the  Landolt  ring,  identification  occurs  if  the  visibility  of 
the  ring  gap  exceeds  a  threshold  value;  however,  it  is  assumed  that  the  participant  has  magnified 
the  image  to  a  level  exceeding  this  threshold.  In  this  case,  the  identification  is  a  function  of  the 
visual  and  vehicular  motion  treatments  and  the  interaction.  That  is, 

Vj  =  Ko*am*av*amxv. 

Again,  taking  the  natural  logarithm  of  both  sides,  we  have  the  linear  expression 

ln(VT)  =  Po  +  Pm*c*m  +  Pv*ttv  +  Pmxv*otmxv> 

in  terms  of  the  effects  coding  (a  =  0  or  1 )  for  the  treatments  and  the  corresponding  powers 
(P)  of  t  he  natural  exponential.  Considering  a  discriminant  function  (Dj),  which  is  unity  for  a 
successful  identification  and  zero  for  a  failure  to  identify,  a  stepwise  regression  analysis  of  the 
target  identifications  for  those  targets  detected  is  significant  (adjusted  R-square  =  0.309, 
p  <  .001,  F  =  1 14.131,  df  =  2,  dfe  =  505),  with  significant  coefficients  for  vehicular  motion 
(bm  =  -0.105,/?  <  .002),  and  viewing  with  the  HMD  (bv  =  -0.530,/?  <  .001),  with  the 
interaction  excluded  by  the  analysis.  As  expected,  the  vehicular  motion  and  use  of  the  HMD 
decrease  the  identifications. 

4.5  Correct  Identification 

A  similar  argument  applies  to  the  incidents  of  correct  identification  for  those  targets  identified. 
A  regression  analysis  of  the  correct  identifications  is  significant  (adjusted  R-square  =  0.109, 
p  <  .000,  F  =  46.476,  df =  1,  dfe  =  370),  with  significant  coefficients  for  the  vehicular  motion 
(bm  =  -.306,/?  <  .001)  but  not  for  viewing  with  the  HMD,  which  is  excluded.  As  expected,  the 
vehicular  motion  decreases  the  incidences  of  correct  identification. 


5.  Mental  Workload  Measures 


While  the  task  model  developed  previously  describes  the  task  performance,  the  mental  workload 
is  of  interest  since  it  can  influence  performance.  Over  long  periods  of  time,  excessive  mental 
loading  can  lead  to  fatigue  and  consequently  slow  responses  and  increase  errors.  Furthermore, 
increased  mental  workload  can  result  in  a  loss  of  SA  for  other  problems,  which  can  impact  future 
decision  making.  Discussed  are  the  mental  workload  measures  as  determined  by  the  relation 
between  the  perceived  task  workload  and  the  attention  allocation,  S  A,  and  motion  sickness.  The 
relationship  among  the  measures  is  analyzed,  and  the  dependency  of  the  perceived  workload  on 
the  other  mental  workload  measures  is  determined.  We  start  with  a  discussion  of  task-directed 
behavior  and  an  interpretation  of  the  workload  measures  in  behavioral  terms.  This  approach 
follows  a  procedure  developed  to  model  workload  for  indirect  vision  driving  (Smyth,  2002a). 
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5.1  Workload  Measures 

The  mental  workload  measures  used  in  this  study  are  the  product  of  the  previously  mentioned 
field  experiment  (Smyth,  2002b),  during  which,  subjective  evaluations  were  collected  as 
completed  questionnaires  from  the  participants  at  the  end  of  each  block  of  trial  runs. 

5.1.1  Mental  Workload  Questionnaires 

The  questionnaires  applied  in  the  field  experiment  are  as  follow. 

5.1. 1.1  NASA  TLX 

The  National  Air  and  Space  Administration  (NASA)  TLX  workload  questionnaire  (Hart  & 
Staveland,  1988)  is  used  for  rating  the  perceived  workload  in  terms  of  task  demand  and 
interaction.  The  NASA-TLX  is  a  multidimensional  rating  procedure  for  the  subjective 
assessment  of  workload.  The  developers  of  the  questionnaire  have  defined  workload  as  a 
hypothetical  construct  that  represents  the  cost  incurred  by  the  human  operator  to  achieve  a 
specific  performance  level.  The  construct  is  composed  of  behavioral,  performance, 
physiological,  and  subjective  components  that  result  from  the  interaction  between  a  specific 
individual  and  the  demands  imposed  by  a  particular  task.  The  questionnaire  consists  of  six  9- 
point  (0  to  9)  bipolar  scales  for  rating  the  mental,  physical,  and  temporal  demands  of  the  task  and 
the  effort,  performance,  and  frustration  of  the  participant  (Charleton,  1996). 

5. 1.1.2  Allocation  of  Task  Attention 

The  questionnaire  is  used  for  rating  the  allocation  of  attention  to  the  visual,  auditory,  cognitive, 
and  motor  processing  channels  of  the  human  operator  according  to  loading  factors  (McCracken 
&  Aldrich,  1984).  These  loading  factors  are  used  in  task  analysis  workload  (TAWL)  simulations 
(Allender,  Salvi,  &  Promisel,  1998).  The  questionnaire  consists  of  a  set  of  four  7-point  (0  to  7) 
bipolar  scales  for  rating  the  attention  loading  on  each  channel,  with  verbal  anchors  for 
corresponding  activities  overlaid  on  the  scales. 

5. 1.1. 3  SART 

The  SART  questionnaire,  which  is  used  for  rating  the  SA  (Taylor,  1988, 1989;  Taylor  &  Selcon, 
1994;  Selcon,  Taylor,  &  Koritsas,  1991)  was  designed  to  measure  subjective  ratings  of  non- 
attentional  factors  such  as  domain  knowledge  or  schemata  and  experience,  the  cognitive  nature 
of  the  information  received  while  the  task  is  being  performed,  and  the  workload  needed  to 
process  the  information.  The  questionnaire  consists  of  ten  7-point  (1  to  7)  bipolar  scales  for 
rating  the  dimensions  of  assessment  demand,  ability,  and  knowledge.  In  this  study,  we  refer  to 
SA  as  the  participant’s  awareness  of  the  location  of  his  experimental  vehicle  on  the  targeting 
course  during  a  trial  run. 


5. 1.1.4  Motion  Sickness 

The  motion  sickness  questionnaire  is  used  for  the  subjective  estimation  of  motion  sickness 
(Kennedy,  Lilienthal,  Berbaum,  Baltzley,  &  McCauley,  1989).  The  questionnaire  lists  4-point  (0 
to  3)  bipolar  rating  scales  consisting  of  verbal  descriptors  for  the  16  symptoms  such  as  general 
discomfort,  eye  strain,  dizziness,  and  nausea.  Based  on  data  from  a  factor  analysis  of  simulator 
sickness  experiences,  a  procedure  has  been  developed  (Kennedy,  Lane,  Lilienthal,  Berbaum,  & 
Hettinger,  1992)  for  reducing  the  scores  to  subscales  for  the  symptomatic  components  of 
oculomotor  stress  (eye  strain),  nausea,  and  disorientation,  and  in  turn,  a  measure  of  total  severity. 

5.1.2  Behavioral  Interpretation  of  Measures 

The  perceived  workload  is  the  result  of  the  attention  devoted  to  the  separate  mental  functions 
needed  in  targeting.  These  are  the  needs  for  allocating  attention  to  the  targeting  task,  maintaining 
SA,  and  monitoring  for  motion  sickness.  The  perceived  workload  is  the  mental  workload 
imposed  on  the  participant  in  the  experiment.  The  workload  measure  consists  of  demand  and 
interaction  dimensions.  Although  demand  dimensions  separately  rate  the  perceived  levels  of  the 
perceptual,  mental,  and  physical  loading  experienced  by  the  participant,  the  interaction 
dimensions  rate  the  perceived  success  of  the  control  applications.  In  the  following,  an  argument 
is  presented  for  the  measures  describing  different  mental  functions  that  impact  perceived 
workload.  Perceived  workload  is  interpreted  as  the  product  of  the  other  measures:  task  attention 
loading,  associative  processing  for  SA,  and  the  monitoring  of  the  somatic  state  of  motion 
sickness.  This  interpretation  follows  from  a  review  of  the  questionnaires,  the  results  of  which  are 
presented  as  follows: 

5. 1.2.1  NASA  TLX 

The  perceived  workload  is  the  workload  imposed  on  the  operator  by  his  or  her  participation  in 
the  task.  While  the  demand  dimensions  refer  to  the  perceptual,  mental  and  physical  components 
of  task  involvement,  the  interaction  dimensions  refer  to  the  levels  of  evaluation  needed. 

5. 1.2.2  Attention  Allocation 

The  ratings  describe  the  attention  that  needs  to  be  applied  to  the  human  information  processors 
(i.e.,  channels)  to  perform  the  task.  These  are  the  vision  and  auditory  perceptual  processors,  the 
cognitive  processor  and  the  psychomotor  processor.  While  the  perceptual  processors  concern 
association  and  encoding  of  stimuli,  the  cognitive  processor  evaluates  and  selects  actions  through 
working  memory.  The  psychomotor  processor  executes  the  actions  selected.  However,  a  review 
of  the  rating  anchors  used  in  this  study  suggests  that  the  ratings  are  limited  to  the  performance  of 
well-learned  sequential  tasks  such  as  selecting  a  radio  frequency  while  flying  an  aircraft 
(McCracken  &  Aldrich,  1984).  In  general,  the  working  memory  of  the  cognitive  processor  can 
be  involved  in  more  engaging  tasks,  such  as  selecting  courses  of  action  from  long-term  memory 
(Warwick,  Mcllwaine,  Hutton,  &  McDermott,  2001). 
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5. 1.2.3  SART 

The  questionnaire  judges  the  ease  with  which  environmental  cues  can  be  assimilated  into  the 
mental  model  of  the  task  structure.  The  demand  dimensions  describe  the  perceptual  nature  of  the 
cues,  the  number  of  variables,  the  complexity  of  the  relations  among  them,  and  how  quickly  the 
relations  are  changing.  These  dimensions  determine  the  perceptual  attention  needed  to  track  the 
cues  and  associate  them  with  the  elements  of  the  task  mental  model.  The  supply  dimensions 
describe  the  mental  capacity  available  to  process  the  perceived  cues.  This  capacity  is  determined 
by  the  alertness,  the  spare  mental  capacity  available,  the  concentration  demanded  by  the  task 
being  performed,  and  the  distribution  of  attention  among  competing  tasks.  The  understanding 
dimensions  describe  the  ease  with  which  these  cues  are  assimilated  with  the  mental  model  as 
determined  by  the  information  associated  with  the  cues  and  with  prior  experience.  Essentially,  an 
environmental  cue  that  cannot  be  easily  assimilated  into  the  task  mental  model  will  provoke  a 
focus  of  attention  onto  the  cognitive  processor  for  further  study  and  evaluation. 

5. 1.2.4  Motion  Sickness 

The  questionnaire  rates  the  attention  dedicated  to  the  somatic  channel  that  is  induced  by  the 
occurrence  of  motion  sickness  symptoms.  As  has  been  noted,  while  mild  motion  sickness  results 
in  discomfort,  extreme  cases  can  arrest  operant  behavior,  causing  the  primary  task  to  be 
momentarily  discontinued  while  full  attention  is  focused  on  the  somatic  state. 

5.2  Analysis  of  Mental  State  Measures 

We  analyzed  the  mental  workload  measures  for  significant  groupings  of  the  measure  dimensions. 
It  is  argued  that  the  application  of  a  factorial  analysis  to  the  mental  workload  measures 
constitutes  a  cognitive  loading  space,  which,  following  rotation,  forms  a  “skills-rules- 
knowledge”  information  processing  space.  The  dependency  of  the  perceived  workload 
dimensions  on  the  SRK  components  is  analyzed  and  discussed. 

5.2.1  Statistical  Relationships  Among  Measures 

Table  8  is  a  correlation  matrix  for  the  dimensions  of  the  mental  workload  measures.  Because 
many  of  the  distributions  are  non-parametric,  the  correlation  entries  are  computed  with  the 
bivariate  Spearman-rho  statistic.  The  table  shows  in  bold  lettering  the  entries  that  are  significant 
at  the  p  <  0.01  level  (two-tailed  test).  Using  this  level  of  probability  as  a  lower  boundary,  we 
consider  first  the  interrelations  among  the  dimensions  of  the  measures  and  in  turn,  the  intra¬ 
relations  among  dimensions  between  measures. 
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Table  8.  Spearman-rho  Correlation  Matrix  for  Mental  Workload  Measures 
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Note:  Bold  correlation  entries  statistically  significant  at  the  p  <  0.01  level  (two-tailed  test). 


5.2.1. 1  Perceived  Workload 

All  dimensions  of  perceived  workload  are  significantly  correlated  with  each  other.  In  turn,  some 
workload  dimensions  are  correlated  with  those  of  attention  loading,  SA,  and  the  nausea  symptom 
of  motion  sickness.  In  particular,  all  workload  demand  dimensions  are  correlated  with  the  SART 
demand  dimensions,  all  are  correlated  with  some  supply  dimensions,  and  mental  and  temporal 
demand  are  correlated  with  the  familiarity  dimension  and  nausea.  Further,  the  workload  mental 
demand  is  correlated  with  visual  loading  for  detection  and  with  physical  demand  with  the 
auditory  channels.  Similar  comments  apply  to  the  workload  interaction  dimensions,  with  all 
these  dimensions  correlated  with  some  of  the  SART  demand,  supply,  and  understanding 
dimensions,  and  with  the  nausea  symptom.  Effort  is  also  correlated  with  visual  identification, 
and  frustration  is  correlated  with  auditory  identification. 

5.2.1. 2  Attention  Allocations 

The  visual  channels  are  correlated  with  the  respective  cognitive  channels,  the  identification 
cognitive  channel  with  the  visual  detection,  and  all  detection  channels  with  the  respective 
identification  channels.  In  turn,  the  auditory  channels  are  correlated  with  the  SART  arousal,  the 
auditory  detection  also  with  the  mental  capacity  and  familiarity,  and  the  visual  detection  with 
familiarity.  However,  none  of  the  attention-loading  dimensions  are  correlated  with  the  motion 
sickness  symptoms. 

5.2.1. 3  Situational  Awareness 

All  demand  dimensions  are  correlated  with  each  other,  as  are  all  understanding  dimensions. 
Arousal  is  correlated  with  the  other  supply  dimensions  and  concentration  with  attention  division. 
All  demand  dimensions  are  correlated  with  some  supply  dimensions,  and  some  demand 
dimensions  are  correlated  with  understanding  dimensions.  Arousal  correlates  with  information 
quantity  and  quality,  concentration  with  familiarity,  and  attention  division  with  information 
quantity.  All  supply  dimensions  except  arousal  are  correlated  with  some  sickness  symptoms,  and 
familiarity  with  the  oculomotor  symptom. 

5.2.1.4  Motion  Sickness 

All  symptoms  are  correlated  with  each  other. 

In  summary,  most  of  the  measure  dimensions  from  this  study  are  correlated  within  a  measure  and 
across  measures.  The  workload  dimensions  are  correlated  among  themselves  and  with 
dimensions  of  attention  loading  and  SA  and  with  sickness  symptoms.  In  turn,  the  attention 
loading  factors  are  correlated  with  SART  dimensions  but  not  with  sickness  symptoms.  Finally, 
SART  dimensions  are  correlated  with  themselves  and  with  the  sickness  symptoms. 
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5.2.2  Cognitive  Component  Space 

Perceived  workload  is  interpreted  as  the  weighted  sum  of  the  attention  to  (a)  the  perceptual, 
cognitive,  and  psychomotor  task-level  processors;  (b)  the  perceptual  association  and  cognitive 
processing  of  situational  cues;  and  (c)  monitoring  the  somatic  channel  as  induced  by  motion 
sickness.  However,  as  has  been  noted  before,  the  task  allocation,  SART,  and  motion  sickness 
dimensions  are  intra-correlated.  For  this  reason,  the  derivation  of  a  unique  expression  for  the 
workload  by  regression  analysis  with  the  stepwise  method  is  difficult  because  of  the  collinearity 
between  these  measures.  One  technique  commonly  used  is  principal  component  expansion  to 
reduce  the  measures  to  an  orthogonal  representation  of  the  experimental  variance  and  to  perform 
a  regression  analysis  on  the  resulting  basic  set  (Neter,  Kutner,  Nachtsheim  &  Wasserman,  1996). 

Figure  4  shows  a  factorial  component  loading  diagram  for  the  task  allocation,  SART,  and  motion 
sickness  measures,  following  reduction  to  three  components  with  a  factor  analysis  with  principal 
component  expansion  as  the  extraction  method  (55.53%  total  variance  explained),  and  Varimax 
rotation  with  Kaiser  normalization.  Because  of  the  high  inter-correlation  between  dimensions, 
the  SART  dimensions  for  demand  and  understanding  were  reduced  to  the  corresponding  sums, 
and  the  symptoms  were  reduced  to  the  total  motion  sickness.  However,  the  SART-supply  and 
task  allocations  show  low  inter-correlation,  and  the  separate  dimensions  were  used  in  the 
analysis.  Table  9  lists  the  rotated  component  matrix  for  the  factor  analysis. 

Table  9.  Component  Matrix 


Factorial  Component 

Measure  First  Second  Third 


SART 


Demand 

0.842 

-.115 

0.092 

Arousal 

0.594 

-.072 

0.603 

Capacity 

0.101 

-.166 

0.669 

Concentration 

0.733 

-.094 

0.227 

Division 

0.846 

-.082 

0.078 

Understanding 

0.033 

0.005 

0.606 

Motion  sickness 

Total  severity 

0.583 

-.087 

-.335 

Detection  allocation 

Visual 

-.291 

0.617 

0.133 

Cognitive 

-.128 

0.768 

0.073 

Auditory 

-.023 

0.197 

0.808 

Motor 

-.108 

0.526 

-.026 

Identification  allocation 

Visual 

0.034 

0.743 

0.105 

Cognitive 

0.230 

0.810 

0.016 

Auditory 

0.043 

0.160 

0.805 

Motor 

-.163 

0.426 

-.075 
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5.2.3  SRK  Processing  Structure 

One  way  of  looking  at  the  loading  diagram  is  that  it  forms  a  cognitive  component  space  with 
information  processing  defining  a  cognitive  state  trajectory  through  the  space.  Considering  the 
groupings  of  the  measures,  it  can  be  argued  that  the  component  axes  correspond  to  levels  of 
cognitive  processing.  In  particular,  we  argue  that  the  components  correspond  to  the  skills,  rules, 
and  knowledge  levels  of  reasoning  (Rasmussen,  1983,  1986,  1993).  In  Figure  4,  Component  2 
corresponds  to  the  skills  level,  Component  1  to  the  rules,  and  Component  3  to  knowledge 
reasoning.  This  interpretation  is  based  on  the  following  argument. 


Figure  4.  Cognitive  Component  Space. 

Notice  that  the  SART-understanding  sum,  which  implies  knowledge,  projects  to  Component  3, 
with  little  projection  to  the  other  two  components.  Similarly,  the  SART-demand  sum  along  with 
the  supply  concentration  and  division  of  attention,  projects  to  Component  1,  with  little  projection 
to  the  other  two  components.  Here,  considering  the  nature  of  the  dimensions,  cognitive  demand 
implies  rule-based  analysis.  Finally,  visual  and  cognitive  attention,  which  imply  task  focus,  are 
predominantly  aligned  with  Component  2,  while  SART  and  motion  sickness,  which  imply 
internal  focus,  have  little  projection  to  this  component.  However,  task  focus  in  turn  implies  skill- 
based  behavior. 
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This  interpretation  is  supported  by  the  fact  that  the  projections  of  the  attention  allocation 
dimensions  are  clustered  about  the  intersection  of  Components  1  and  3,  implying  that  little  rule- 
based  or  knowledge-based  reasoning  is  used  in  the  allocation  dimensions.  This  is  true  of  all  but 
the  two  auditory  dimensions,  which  are  clustered  with  the  SART-understanding  sum,  along  with 
the  SART-supply  dimension  of  limited  mental  capacity.  Notice  that  all  SART  values  have 
minimal  projection  to  Component  2,  implying  little  skilled  behavior.  In  turn,  Figure  4  shows 
motion  sickness  with  none  of  the  skills  of  Component  2,  negative  knowledge  or  understanding 
from  Component  3,  and  a  direct  rule  response  for  Component  1. 

5.2.4  Total  Workload 

Following  the  argument  just  given,  the  perceived  workload  is  interpreted  as  a  weighted  sum  of 
the  attention  applied  to  the  three  levels  of  reasoning.  A  correlation  of  the  perceived  workload 
with  the  factorial  components  via  the  non-parametric  bivariate  Spearman-rho  statistic  for  a  two- 
tailed  test  shows  that  the  total  TLX  is  significantly  correlated  with  the  first  (R2  =  0.663, 
p  <  0.001,  N  =  64)  and  second  (R2  =  -.411,/?  =  0.001,  N  =  64)  components  but  not  the  third.  The 
implication  is  that  the  total  workload  is  determined  by  the  attention  demanded  by  the  task 
allocation  and  the  rules  processing  but  not  by  knowledge  processing  for  this  task. 

Applying  curve  estimation  techniques  for  relating  the  workload  to  the  factorial  components,  the 
total  workload  is  described  by  a  cubic  polynomial  with  a  constant  term  in  the  first  component 
and  a  linear  polynomial  with  a  constant  term  in  the  second  but  not  at  all  in  the  third.  A  linear 
regression  analysis  of  the  total  workload  as  a  function  of  the  powers  of  the  first  and  second 
factorial  components  with  the  entry  method  (Pedhazur,  1982)  is  statistically  significant  (adjusted 
R2  =  0.438,  p  <  .001,  F  =  13.256,  df  =  4,  dfe  =  59);  however,  none  of  the  coefficients  are 
significant.  The  entry  method  produces  residuals  that  are  uniformly  distributed  across  component 
values. 

The  results  of  the  analysis  suggest  that  perceived  workload  for  this  study  is  a  product  of  skill- 
and  rule-based  reasoning  but  not  knowledge-based  reasoning.  In  particular,  the  total  workload 
index  (TTLX)  is  given  by 

TTLX  =  -  0.161 

-0.165*SKILLS 

+  1.710*RULES  -  3.1 19E-02*RULES2  +  2.294E-04*RULES3,  (1) 

in  terms  of  the  first  (“RULES”)  and  second  (“SKILLS”)  factorial  components,  with  the 
coefficients  in  scientific  notation.  The  factorial  components  are  expressed  as  a  weighted  sum  of 
the  measures  used  in  the  analysis,  and  the  corresponding  weights  are  listed  in  Table  9.  Following 
Equation  (1),  the  isobars  of  constant  workload  form  a  family  of  curvilinear  lines  in  the  skills- 
rules  plane  (defined  by  Components  1  and  2).  These  isobars  are  perpendicular  to  the  locus  of 
increasing  workload,  which,  for  low  component  values,  is  practically  at  a  -5°  angle  to  the  rules 
axis  (Component  1),  as  shown  in  Figure  4. 
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5.2.5  Statistical  Values  of  Components 

For  modeling  workload,  the  statistical  contributions  of  the  factorial  components  to  the  workload 
are  of  interest.  Table  10  lists  the  averages  and  standard  deviations  for  the  contributions  to  the 
total  workload  from  the  first  two  components  predicted  with  Equation  (1),  as  a  function  of  the 
experimental  treatments  collapsed  across  direct  and  HMD  viewing.  A  study  of  the  statistics  for 
the  contribution  from  the  “SKILLS”  component  shows  that  the  direct  and  indirect  distributions 
for  the  contributions  from  the  component  are  different;  in  particular,  the  mean  for  one 
distribution  is  not  bound  by  the  95%  confident  interval  for  the  mean  of  the  other.  Similarly,  the 
direct  and  indirect  distributions  for  the  contributions  from  the  “RULES”  component  are 
different;  in  particular,  the  mean  for  one  distribution  is  not  bound  by  the  95%  confident  interval 
for  the  mean  of  the  other.  The  implication  is  that  the  significantly  statistical  change  in  workload 
with  visual  treatment  is  caused  by  the  changes  in  both  the  SKILLS  and  RULES  components. 


Table  10.  Significant  Statistics  for  Total  Workload  Contributions  From  The  First  Two  Factorial 
Components 


Component 

N 

Direct 

Average 

Treatment 

SD 

N 

HMD 

Average 

SD 

SKILLS 

32 

14.076 

17.519 

32 

-1.285 

0.807 

RULES 

32 

19.160 

6.577 

32 

26.682 

7.882 

5.2.5. 1  SKILLS  Component 

A  study  of  Table  9  shows  that  the  major  sources  of  Component  2  are  the  task  allocated  attention 
to  the  visual,  cognitive,  and  motor  channels  for  the  detection  and  identification  tasks.  Table  7 
lists  the  statistics  for  these  measures  as  a  function  of  the  experimental  treatments.  The  table 
shows  that  the  average  loading  for  the  detection  task  psychomotor  channel  is  significantly 
increased  by  the  HMD  viewing.  In  contrast,  the  average  loading  values  for  the  visual  and 
cognitive  channels  are  not  significantly  changed  by  the  treatments  for  both  tasks.  This  is  true  also 
for  the  identification  psychomotor  channel.  Thus,  the  loading  on  the  detection  task  psychomotor 
channel  is  the  primary  source  of  the  significant  change  in  the  SKILLS  component  with  viewing 
treatment. 

5.2.5. 2  RULES  Component 

A  study  of  Table  9  shows  that  the  major  sources  of  Component  1  are  the  SART-demand,  the 
SART  concentration  and  division  dimensions,  and  motion  sickness  severity.  Table  6  lists  the 
statistics  for  these  measures  as  a  function  of  the  experimental  treatments.  The  table  shows  that 
the  average  for  the  SART  demand  is  significantly  greater  for  the  HMD  while  the  average  for 
motion  sickness  is  significantly  greater  for  the  HMD  in  the  moving  vehicle.  These  variables  are 
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the  sources  of  the  increase  in  the  workload  contribution  from  Component  1  for  the  indirect  vision 
treatment. 

5.2.6  Expected  Total  Workload 

Using  Equation  (1)  for  the  total  perceived  workload  given  previously,  we  may  compute  the 
average  expected  values  from  the  data  for  each  treatment.  Figure  5  is  a  set  of  bar  column  plots  of 
the  average  predicted  total  workload  as  a  function  of  treatment  conditions.  The  figure  shows  that 
the  average  predicted  values  for  the  direct  viewing  treatment  are  within  or  close  to  the  95% 
confidence  interval  for  the  statistically  significant  average  of  13.966  for  that  treatment.  Similarly, 
the  average  predicted  values  for  the  HMD  viewing  treatments  designated  by  vehicular  motion, 
are  within  or  close  to  the  95%  confidence  intervals  for  the  statistically  significant  averages  of 
26.43 1  for  the  stationary  vehicle  and  29.769  for  the  moving  vehicle. 


Figure  5.  Average  Predicted  Total  Workload  for  Treatment  Conditions. 

5.2.7  Workload  Dimensions 

The  relations  of  the  perceived  workload  dimensions  to  the  factorial  components  are  of  interest. 
In  this  section,  we  present  the  results  of  statistical  analyses  for  the  separate  dimensions  of 
perceived  workload. 

5. 2.7.1  Mental  Demand 

A  correlation  analysis  with  the  non-parametric  bivariate  Spearman-rho  statistic  for  a  two-tailed 
test  shows  that  the  dimension  is  significantly  correlated  with  the  first  (R2  =  0.583,/?  <  0.001, 

N  =  64)  and  second (R2  =  -0.343,/?  =  0.006,  N  =  64)  components  but  not  the  third.  When  we 
apply  curve  estimation  techniques,  the  mental  demand  is  described  by  a  cubic  polynomial  with  a 
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constant  term  in  the  first  component  and  a  linear  term  in  the  second.  A  linear  regression  analysis 
of  the  dimension  as  a  function  of  the  powers  of  the  factorial  components  via  the  entry  method  is 
statistically  significant  (adjusted  R2  =  0.392 ,/?  <  .001,  F  =  1 1 .160,  df  =  4,  dfe  =  59);  however, 
none  of  the  coefficients  are  significant.  The  unstandardized  coefficients  for  the  analysis  are 
b0  =  -0.020,  bi  =  0.197,  b2  =  -5.867E-04,  and  b3  =  -5.550E-06,  for  the  first  component,  and 
bi  =  -2.187E-02  for  the  second,  where  the  indices  match  the  component  power.  The  entry 
method  produces  residuals  that  are  uniformly  distributed  across  component  values.  The  results  of 
the  analysis  suggest  that  mental  demand  for  this  study  is  a  product  of  skill-  and  rule-based 
reasoning  but  not  knowledge-based  reasoning. 

52.1.2  Physical  Demand 

The  dimension  is  significantly  correlated  with  the  first  (R2  =  0.389,  p  =  0.001,  N  =  64) 
component  but  not  the  second  or  third.  The  physical  demand  is  described  by  a  cubic  polynomial 
with  a  constant  term  in  the  first  component.  A  linear  regression  analysis  as  a  function  of  the 
powers  of  the  first  factorial  component  is  statistically  significant  (adjusted  R  =  0.272 ,p  <  0.001, 
F  =  8.855,  df  =  3,  dfe  =  60);  however,  none  of  the  coefficients  are  significant.  The  un¬ 
standardized  coefficients  for  the  analysis  are  bo=  -1.155,  bi  =  0.271,  b2  =  -7.014E-03,  and 
b3  =  6.517E-05.  The  results  of  the  analysis  suggest  that  physical  demand  is  a  product  of  rule- 
based  reasoning  but  not  skills  or  knowledge. 

5.2.7.3  Temporal  Demand 

The  dimension  is  significantly  correlated  with  the  first  (R2  =  0.462,  p  <  0.001,  N  =  64) 
component  but  not  the  second  or  third.  The  dimension  is  described  by  a  cubic  polynomial  with  a 
constant  term  in  the  first  component.  A  linear  regression  analysis  is  statistically  significant 
(adjusted  R2  =  0.21 6,/?  =  0.001,  F  =  6.794,  df  =  3,  dfe  =  66);  however,  none  of  the  coefficients 
are  significant.  The  unstandardized  coefficients  for  the  analysis  are  bo=  -0.161,  bi  =  0.310, 
b2  =  -5.479E-03,  and  b3  =  3.667E-05.  The  entry  method  produces  residuals  that  are  uniformly 
distributed  across  component  values.  The  results  of  the  analysis  suggest  that  the  temporal 
demand  is  a  product  of  rule-based  reasoning  but  not  skills  or  knowledge. 

5 .2.7.4  Effort 

The  dimension  is  significantly  correlated  with  the  first  (R2  =  0.478,/?  <  0.001,  N  =  64)  and 
second  (R2  =  -0.408,/?  =  0.001,  N  =  64)  components  but  not  the  third.  The  dimension  is 
described  by  a  cubic  polynomial  with  a  constant  term  in  the  first  component  and  a  quadratic 
polynomial  in  the  second.  A  linear  regression  analysis  is  statistically  significant  (adjusted 
R2  =  0.293,/?  <  0.001,  F  =  6.229,  df  =  5,  dfe  =  58);  however,  none  of  the  coefficients  are 
significant.  The  un-standardized  coefficients  for  the  analysis  are  bo  =  0. 147  for  the  constant, 
bi  =  0.509,  b2  =  -1.299E-02,  and  b3  =  1.033E-04  for  the  first  component,  and  bi  =  -0.107  and 
b2  =  3.908E-05,  for  the  second.  The  entry  method  produces  residuals  that  are  uniformly 


31 


distributed  across  component  values.  The  results  suggest  that  the  effort  is  a  product  of  the  skill- 
and  rule-based  reasoning  but  not  knowledge. 

5.2.7.5  Performance 

The  dimension  is  significantly  correlated  with  the  first  (R2  =  0.578,  p  <  0.001,  N  =  64) 
component  but  not  the  second  or  third.  The  performance  is  described  by  a  cubic  polynomial  with 
a  constant  term  in  the  first  component.  A  linear  regression  is  statistically  significant  (adjusted 
R2  =  0.374,  p  <  0.001,  F  =  13.571,  df  =  3,  dfe  =  60);  however,  none  of  the  coefficients  are 
significant.  The  un-standardized  coefficients  for  the  analysis  are  b0=  0.302,  bj  =  0.159, 
b2  =  5.537E-04,  and  b3  =  -1.788E-05.  The  entry  method  produces  residuals  that  are  uniformly 
distributed  across  component  values.  The  results  of  the  analysis  suggest  that  performance  is  a 
product  of  rule-based  reasoning  but  not  skills  or  knowledge. 

5.2. 7.6  Frustration 

The  dimension  is  significantly  correlated  with  the  first  (R2  -  0.392,  p  =  0.001,  N  =  64) 
component  but  not  the  second  or  third.  The  dimension  is  described  by  a  cubic  polynomial  with  a 
constant  term  in  the  first  component.  A  linear  regression  analysis  is  statistically  significant 
(adjusted  R2  =  0.181,/?  =  .002,  F  =  5.627,  df  =  3,  dfe  =  60);  however,  none  of  the  coefficients  are 
significant.  The  un-standardized  coefficients  for  the  analysis  are  b0=  0.061,  bi  =  0.282, 
th  =  -5.910E-03,  and  b3  =  4.914E-05.  The  entry  method  produces  residuals  that  are  uniformly 
distributed  across  component  values.  The  results  of  the  analysis  suggest  that  the  frustration  is  a 
product  of  rule-based  reasoning  but  not  skills  or  knowledge. 

In  summary,  the  statistical  analyses  for  the  separate  dimensions  of  perceived  workload  as  a 
function  of  the  factorial  components  interpreted  as  cognitive  levels  of  reasoning  agree  with  the 
experimental  results  reported  in  the  literature.  This  is  true  since  they  demonstrate  the  increase  in 
workload  demand  and  interaction  with  use  of  the  HMD  (Smyth,  2002b).  Application  of  the  Holm 
simultaneous  testing  procedure  (Neter,  Kutner,  Nachtsheim,  &  Wasserman,  1996)  for  control  of 
the  type  I  error  shows  that  the  separate  regression  analyses  listed  previously  for  the  total 
workload  and  dimensions  satisfy  a  family-wise  0.05  alpha  level.  The  implication  is  that  the 
separate  analyses  are  not  statistically  significant  by  chance  alone  because  of  the  multiple  testing. 

5.2.8  Implications  for  Task  Modeling 

The  results  suggest  that  perceived  workload  is  a  function  of  what  is  being  interpreted  as  skill- 
and  rule-based  reasoning  but  not  of  knowledge-based  reasoning.  This  is  reasonable  since 
knowledge  of  the  target  range  and  experimental  procedure  was  learned  by  the  participants  at  the 
start  of  the  experiment  and  did  not  vary  with  treatments.  It  is  interesting  that  while  both  the  skill- 
and  rule-based  reasoning  are  different  by  treatment,  the  attention  allocation  ratings  used  in  this 
study  appear  to  map  to  the  skills  level.  However,  intense  cognitive  processing  is  necessary  to 
support  the  rules  processing  and  maintain  the  mental  model  that  is  needed  for  range  targeting. 
One  possibility  is  that  rating  forms  did  not  include  activities  as  anchors  that  are  pertinent  to  this 
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level  of  reasoning.  In  the  next  section,  we  propose  a  micro-activity  model  for  the  skill-,  rule-,  and 
knowledge-based  reasoning,  which  may  be  applied  to  the  study  performance.  The  models  are 
used  to  show  that  the  differences  in  workload  between  treatments  may  be  explained  by  the  rule- 
based  reasoning  that  is  needed  to  monitor  the  motion  sickness  during  indirect  vision  in  a  moving 
vehicle.  Further,  the  models  are  used  to  determine  the  relations  among  the  ratings  for  the 
different  measures  and  particularly,  the  relation  of  the  perceived  workload  index  to  the  ratings  of 
task  allocation  of  attention. 


6.  Micro-Activity  Time  Line  Model 


In  this  section  of  the  report,  we  describe  micro-activity  time  line  models  for  direct  and  indirect 
vision  targeting.  In  keeping  with  the  literature,  human  response  is  assumed  to  be  directed  by 
micro-level  activities  that  occur  within  cortical  based  processors,  which  consist  of  perceptual 
processors,  a  cognitive  processor  interfacing  with  memory,  and  a  motor  processor  (Card, 

Morgan,  &  Newell,  1983).  The  processing  times  are  assumed  to  be  on  the  order  of  100 
milliseconds  (Card,  Morgan,  &  Newell,  1983),  with  a  demand  loading  corresponding  to  the 
attention  needed  to  process  the  information  for  the  task  (McCracken  &  Aldrich,  1984). 
Furthermore,  loading  is  assumed  to  be  increased  by  interference  that  occurs  within  processors 
during  the  performance  of  concurrent  tasks  (Little  et  al.,  1993). 

Here,  skilled  reasoning  is  interpreted  as  a  sequence  of  over-learned,  automatic  activities 
performed  in  a  pipelined  manner  between  connected  processors,  from  perceptual  to  cognitive 
and  then  to  motor  action.  In  contrast,  rule-based  reasoning  is  interpreted  as  being  a  cognitive 
processor  activity  of  an  evaluation  nature,  particularly  of  an  “if-then”  production  rule.  Finally, 
knowledge-based  reasoning  is  interpreted  as  a  cognitive  activity  involving  memory  recall  and 
processing.  The  development  of  the  models  is  guided  by  the  results  of  the  workload  analysis,  that 
is,  knowledge-based  reasoning  remains  consistent  across  treatments,  while  aspects  of  the  skill- 
and  rule-based  reasoning  are  changed  by  the  indirect  vision.  Further,  the  activities  are  restricted 
to  the  experimentally  derived  task  times  for  the  treatments.  We  describe  first  the  model  for 
stationary,  direct  vision  without  sound  localization  and  then  the  model  for  indirect  vision  with 
the  HMD  in  the  moving  vehicle  with  sound  localization.  These  two  treatments  are  extreme  cases 
for  the  reported  perceived  workload. 

6.1  Case  I:  Direct  Vision,  Stationary  Vehicle,  Without  Sound  Localization 

Table  1 1  is  a  chart  of  a  micro-activity  time  line  for  Case  I.  The  trial  status  is  listed  in  the  “mode” 
column  as  a  function  of  the  trial  times  listed  in  the  “time”  column  in  milliseconds.  The  actions  of 
the  experimenter  and  the  range  control  operator  are  listed  in  the  “tester”  and  “range”  columns, 
respectively.  The  micro-activities  performed  by  the  participant  are  listed  in  an  activity  column, 
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7230  move  flag 
7730  flag  up 

End  run  7900  c/c  key 

End  recall _ 84 1 0 _ Total  workload  =  60.464 

Note:  [**]  bracketed  times  concurrent 


with  the  type  of  reasoning  that  the  activity  uses  indicated  in  the  “base”  column  with  the  letter 
“K”  for  knowledge-based,  “R”  for  rules-based,  and  “S”  for  skill-based  reasoning.  The  cortical 
processors  involved  in  these  activities  are  listed  in  the  next  set  of  columns.  The  time  to  perform 
each  activity  is  listed  in  the  interval  column,  followed  by  the  attention  loading  and  the  equivalent 
verbal  anchor.  Finally,  the  workload  incurred  is  listed  as  the  product  of  attention  loading  x  the 

processor  time  interval. 

According  to  the  experimental  results  for  Case  I,  the  time  to  detect  the  target  averaged 
3.367  seconds  and  that  to  identify  the  orientation  of  the  Landolt  C-ring  attached  to  the  face  of  the 
target  was  4.390  seconds  (Smyth,  2002b).  These  times  are  measured  from  the  facility  control  key 
actions,  and  the  trial  times  listed  in  the  table  are  adjusted  for  time  used  by  the  experimenter  to 
start  the  trial.  As  noted  in  the  activity  columns,  the  participant  at  the  start  is  waiting  for  the  verbal 
command  from  the  experimenter  (“tester”)  to  begin  searching  for  the  target.  The  command  is 
given  about  1  second  after  the  experimenter  has  raised  the  signal  flag.  This  allows  enough  time 
for  the  facility  control  operator  to  respond  by  activating  the  computer  control  key  (170 
milliseconds  [ms]  for  visual  perception  and  motor  response)  and  for  the  target  board  to  be 
completely  raised  by  the  mechanism  (about  0.75  second). 

Upon  hearing  the  command  to  start  the  detection  process,  the  participant  opens  his  eyes  and 
cognitively  confirms  the  audio  input.  This  is  interpreted  as  a  rule-based  behavior  with  the 
cognitive  action  occurring  concurrently  with  the  oculomotor  action.  The  participant  then 
executes  a  sequence  of  skill-based  activities  consisting  of  a  visual  glimpse  followed  by  the 
indexing  of  the  eyes  and  head  across  the  visual  scene.  Since  the  eye  movements  are  saccadic, 
visual  perception  can  only  occur  during  the  consequent  fixation  period.  Concurrent  with  the 
motor  action  is  a  cognitive  confirmation  of  the  target  absence.  The  cognitive  activity  is  assumed 
to  be  pipelined  with  the  previous  visual  perception.  Finally,  in  this  process,  the  participant 
realizes  that  he  has  seen  the  target  and  activates  a  verbal  response.  This  last  behavior  is 
interpreted  as  rule  based.  Following  a  170-ms  delay  for  audio  perception  and  motor  response,  the 
experimenter  takes  500  ms  to  raise  the  signal  flag  to  indicate  detection.  In  turn,  after  another 
1 70-ms  delay  (visual  perception  and  motor  response),  facility  control  has  activated  the  computer 
control  key,  thereby  recording  the  detection  event  and  the  start  of  the  identification  process. 

Concurrent  with  verbal  delivery,  the  participant  recalls  from  memory  his  knowledge  of  the 
procedure  for  the  identification  task.  Since  he  is  switching  his  procedural  mode,  we  estimate  that 
the  time  for  recalling  and  establishing  the  procedural  rules  and  cognitive  reference  frame  is  about 
1.35  seconds  as  reported  in  the  literature  (Card,  Moran,  &  Newell,  1980).  Associated  with  the 
recall  is  an  increase  in  attention  loading  because  of  interference  between  cognition  and  speech 
delivery  (Little  et  al.,  1993).  In  this  model,  the  interference  is  further  aggravated  by  previous 
priming  of  the  cognitive  working  memory  for  judgment  of  the  target  sighting.  The  increase  in 
loading  is  expressed  as  the  sum  of  the  separate  task  loadings  x  a  conflict  factor,  which  is 
commonly  0.5  for  interference  between  the  cognitive  processor  and  the  motor  processor  for 
speech  (Allender,  1998). 
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Following  setup,  the  participant  moves  the  binoculars  to  his  eyes  and  indexes  his  head  in  a  rule- 
based  activity  to  center  the  target  within  the  visual  field.  The  participant  then  proceeds  to  “zoom 
in”  on  the  target  in  a  skill-based  sequence  of  visual  glimpses.  These  glimpses  are  accompanied 
with  concurrent  hand  adjustments  for  scene  magnification,  eye  and  head  adjustments  for 
centering  the  visual  field,  and  a  cognitive  check  of  the  scene.  In  this  case,  the  eyes  are  fixed  on 
the  target,  and  the  eye  movements  track  the  target  as  movements  are  induced  in  the  scene  by  the 
increased  magnification.  These  concurrent  activities  are  automatically  pipelined  to  the  previous 
visual  perception.  Finally,  the  participant,  realizing  that  he  has  seen  the  Landolt  ring  on  the 
target,  reports  the  orientation  as  “ID  up,”  “ID  down,”  “ID  left,”  or  “ID  right”.  This  last  is 
interpreted  as  a  rule-based  behavior.  Again,  concurrent  with  his  verbal  delivery,  the  participant 
recalls  from  memory  his  knowledge  of  the  procedure  for  actions  between  trial  runs,  that  is,  to 
turn  his  head  down  range  and  close  his  eyes.  As  before,  associated  with  the  recall  is  an  increase 
in  attention  loading  because  of  interference  between  cognition  and  speech  delivery.  Again,  the 
interference  is  further  aggravated  by  previous  priming  of  the  cognitive  working  memory  for 
judgment  of  the  target  identification.  Following  a  170-ms  delay  for  audio  perception  and  motor 
response,  the  experimenter  again  takes  500  ms  to  raise  the  signal  flag  to  indicate  identification. 
In  turn,  after  another  170-ms  delay  (visual  perception  and  motor  response),  facility  control  has 
activated  the  computer  control  key  to  record  the  identification  event  and  the  end  of  the  trial. 

The  amount  of  mental  workload  incurred  in  performing  the  activity  is  indicated  in  the  last 
column.  The  workload  increment  for  the  processor  is  the  product  of  the  attention  loading  x  the 
processor  interval  expressed  in  seconds.  The  total  workload  incurred  in  target  detection  and 
identification  is  the  sum  of  the  workload  increments.  For  stationary  direct  vision  without  sound 
localization,  the  total  workload  incurred  is  60.464  in  units  of  attention  loading  x  seconds, 
modeled  over  a  7.450-second  time  line  including  the  final  procedural  recall  for  the  trial  run. 

6.2  Case  II:  Indirect  Vision  With  HMD,  Moving  Vehicle,  With  Sound  Localization 

Table  12  is  a  chart  of  a  micro-activity  time  line  for  Case  II.  The  chart  has  the  same  format  as 
Table  1 1  for  direct  vision,  with  additional  activities  for  the  primary  task  of  target  search  that  are 
associated  with  the  sound  localization  and  facility  mapping  during  vehicle  movement.  While  the 
initial  response  to  the  sound  tone  is  a  rule-based  head  shift  to  the  localizing  direction,  the 
participant  ignores  the  tone  during  the  rest  of  the  trial  and  processes  the  audio  perception  at  a 
skill-based  orientation  loading  level.  This  is  because  following  initial  localization,  the 
participants  preferred  to  continue  the  search  with  the  finer  resolution  of  the  visual  process.  This 
result  is  apparent  in  the  data  reported  in  other  studies  with  target  detection  via  an  HMD  with 
sound  localization  (Nelson  et  al.,  1998).  Note  the  interference  between  the  motor  processor 
during  the  head-turning  response  for  initial  localization  and  the  audio  processing  of  the  tone.  As 
before,  the  increase  in  loading  is  expressed  as  the  sum  of  the  separate  task  loadings  x  a  0.5 
conflict  factor. 
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Vehicular  movement  increases  the  cognitive  load  on  the  participants.  This  is  because  the  scene 
being  searched  is  continually  changing  as  the  vehicle  moves  forward  alongside  the  target  facility. 
Consequently,  in  preparation  for  each  ocular  and  head  shift  during  target  search,  the  participant 
cognitively  rotates  a  mental  map  of  the  facility  to  find  his  next  viewing  point.  Note  that  in 
preparation  for  the  experiment,  the  participant  was  allowed  to  study  the  facility  and  target 
locations  before  the  trials.  This  mental  mapping  process  is  repeated  during  zooming  for  target 
identification,  since  the  movement  of  the  vehicle  tends  to  move  the  viewing  point  past  the  target. 
In  shifting  his  view  to  compensate  for  the  movement,  the  participant  must  again  mentally 
maintain  his  location  on  the  facility.  During  a  visual  glimpse,  the  participant  concurrently  holds 
and  evaluates  a  head  and  eye  orientation  for  effective  acquisition;  these  activities  are  performed 
in  a  pipelined  manner  from  the  previous  images.  Following  visual  acquisition,  the  scene  is 
evaluated  for  the  presence  of  the  target  during  detection  or  whether  the  target  is  centered  during 
identification,  with  the  latter  performed  concurrently  with  the  head  movement  during  the  image 
zooming  process. 

Added  to  Table  12  are  two  columns  for  activities  associated  with  the  processing  of  motion 
sickness  during  vehicle  movement  as  a  second  task.  Associated  with  each  eye  glimpse  is  the 
generation  of  sensory  conflict  between  the  visual  and  sensorimotor  activities,  which  involve  the 
vestibular  system  through  head  movements.  This  sensory  conflict  is  assumed  to  be  perceived  by 
a  somatic  processor,  possibly  with  associated  autonomic  responses  leading  to  pallor,  drowsiness, 
salivation,  sweating,  nausea,  and  finally,  vomiting  in  the  more  severe  cases  of  sickness. 
Concurrent  with  the  somatic  perception  is  a  head  movement  by  a  participant  experienced  with 
the  indirect  vision  system,  to  counter  the  sensations  through  opposing  movements  of  the 
vestibular  system.  However,  the  head  movement  response  is  based  on  the  previous  visual  image, 
and  the  residual  somatic  sensation  is  evaluated  in  a  following  cognitive  activity  at  the  rules  level. 

As  before,  associated  with  the  performance  of  concurrent  tasks  is  an  increase  in  attention  loading 
because  of  interference  within  and  between  processors  (Little  et  al.,  1993).  However,  unlike  the 
model  for  Case  I,  interference  is  a  major  source  of  processor  loading.  Essentially,  we  expect 
interference  within  the  motor  processor  between  head  holding  and  control  adjustment  for  the 
sighting  task  and  head  shifting  to  counter  motion  sickness.  Further,  we  expect  interference  within 
the  cognitive  processor  during  sickness  resolution  because  of  the  previous  priming  for  scene 
mapping.  Similarly,  we  expect  interference  between  the  visual  and  cognitive  processors  during 
resolution  because  of  the  previous  associations  for  scene  mapping.  Finally,  we  expect 
interference  between  the  visual  and  audio  processors  during  sound  localization  cueing.  The 
increase  in  loading  is  expressed  as  the  sum  of  the  separate  task  loadings  x  a  conflict  factor, 
which  is  commonly  0.7  for  interference  within  a  processor  and  0.5  for  interference  between  the 
visual  and  cognitive  processors  (Allender,  1998).  The  following  is  a  more  detailed  description  of 
the  effect  of  interference  on  the  loading  of  the  processors. 

Consider  first  the  detection  task,  particularly  the  “look”  portion  of  the  search  routine.  As  shown 
in  Table  12,  interference  occurs  within  the  motor  processor  during  a  visual  glimpse  because  of 
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the  conflict  between  the  head  and  eye  shift,  which  is  needed  to  fix  the  view  for  the  sighting  task, 
and  the  concurrent  head  shift  used  to  counter  motion  sickness.  Similarly,  interference  occurs 
within  the  cognitive  processor  between  the  visual  and  somatic  perceptions  during  scene 
mapping.  Further,  the  previous  priming  of  the  motor  processor  for  head  adjustment  interferes 
with  indexing  the  eyes  to  the  next  scene  fixation  point.  Similarly,  the  retention  of  the  visual 
perception  in  working  memory  and  the  priming  of  the  cognitive  processor  for  computing  the 
scene  fixation  interferes  with  resolution  of  the  residual  motion  sickness.  Finally,  the  audio 
perception  of  the  tone  interferes  with  the  cognitive  and  motor  processing,  further  aggravating  the 
existing  interference  because  of  the  suppression  of  the  localizing  response.  A  similar  argument 
applies  to  the  head  adjustment  portion  of  the  search  routine,  except  that  here,  the  interference  in 
the  cognitive  processor  is  caused  by  the  previous  priming  for  evaluation  and  judgment.  This 
effect  of  previous  evaluation  applies  also  to  interference  in  the  cognitive  processor  during  target 
verification  following  search  and  to  interference  from  the  audio  tone.  As  with  the  model  for 
Case  I,  there  is  interference  between  the  speech  for  target  detection  and  the  cognitive  recall  of 
the  identification  task;  here  again,  the  previous  priming  for  evaluation  is  expected  to  increase  the 
cognitive  load.  The  tones  during  speech  further  aggravate  the  existing  cognitive  interference. 

Similar  comments  apply  to  the  identification  task.  The  visual  glimpse  has  associated  with  it 
interference  in  the  motor  processor  because  of  conflict  with  the  body  movements  used  in  scene 
centering  during  zooming  and  the  head  shifts  used  to  counter  motion  sickness.  Again, 
interference  occurs  within  the  cognitive  processor  between  the  visual  and  somatic  perceptions 
during  scene  mapping  for  maintaining  the  view  on  the  target.  Further,  the  previous  priming  of 
the  motor  processor  for  head  adjustment  to  maintain  the  view  interferes  with  the  hand  and 
fingers  movements  to  increment  magnification.  The  retention  of  the  visual  perception  in  working 
memory  and  the  priming  of  the  cognitive  processor  for  computing  the  scene  fixation  interferes 
with  resolution  of  the  residual  motion  sickness.  The  audio  perception  of  the  tone  interferes  with 
the  cognitive  and  motor  processing,  further  aggravating  the  existing  interference  because  of  the 
suppression  of  the  localizing  response.  This  effect  of  previous  evaluation  applies  also  to 
interference  in  the  cognitive  processor  during  identity  verification  following  zooming  and  to 
interference  from  the  audio  tone.  There  is  interference  between  the  speech  for  target 
identification  and  the  cognitive  recall  of  the  trial  preparation  task;  here  again,  the  previous 
priming  for  evaluation  is  expected  to  increase  the  cognitive  load.  Also,  the  tones  during  speech 
further  aggravate  the  existing  cognitive  interference. 

Notice  that  the  model  proposed  here  for  Case  II  follows  the  dictates  of  the  data  analysis  of 
workload  presented  before.  While  the  knowledge-based  activities  of  recalling  the  identification 
procedure  rules  and  reference  frame  remain  consistent  across  models,  the  skill-  and  rule-based 
activities  are  changed.  The  increase  in  search  time  for  the  indirect  vision  is  consumed  by  the 
response  to  motion  sickness.  However,  the  addition  of  the  motion  sickness  response  task  has 
greatly  increased  the  rule-based  activities.  Furthermore,  the  vehicle  movement  has  increased  the 
cognitive  rule-based  activities. 
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According  to  the  experimental  results  for  Case  II,  the  time  to  detect  the  target  averaged 
9.054  seconds  and  time  to  identify  the  Landolt  C-ring  orientation  was  7.010  seconds  (Smyth, 
2002b).  These  times  are  measured  from  the  facility  control  key  actions,  and  the  trial  times  listed 
in  the  table  are  adjusted  for  the  time  taken  by  the  experimenter  to  start  the  trial.  The  amount  of 
mental  workload  incurred  in  performing  the  activity  is  indicated  in  the  last  column.  The 
workload  increment  for  the  processor  is  the  product  of  the  attention  loading  x  the  processor 
interval  expressed  in  seconds.  The  total  workload  incurred  in  the  target  detection  and 
identification  task  is  the  sum  of  the  workload  increments.  For  this  model,  the  total  workload 
incurred  is  245.1 14  in  units  of  attention  loading  x  seconds,  modeled  over  a  15. 43 -second  time 
line. 


6.3  Relation  to  Mental  Workload  Measures 

From  the  model  structure,  we  determine  the  relations  of  the  ratings  for  the  micro-activity  time 
line  models  to  the  mental  workload  measures  of  allocation  of  task  attention,  S  A,  motion 
sickness,  and  subjective  stress. 

6.3.1  Allocation  of  Task  Attention 

Table  7  shows  that  the  loading  ratings  reported  by  the  participants  for  the  measures  of  the 
allocation  of  task  visual  and  cognitive  attention  for  the  detection  and  identification  tasks  are 
statistically  independent  of  the  treatments.  While  this  is  also  true  for  the  psychomotor  loading  for 
the  identification  task,  loading  for  the  detection  task  depends  on  the  vision  treatment.  Finally,  the 
audio  channel  loading  depends  on  the  sound  localization  treatment  for  both  tasks.  In  the  SRK 
processing  structure  described  before,  the  ratings  of  allocated  attention  dominate  the  SKILLS 
component.  In  Tables  1 1  and  12  for  the  micro-activity  models,  the  skill  activities  are  applied  to 
the  detection  look  and  identification  zoom  portions  of  the  time  line.  In  particular,  the  sequence  of 
a  perceptual  glimpse,  cognitive  verification,  and  motor  indexing  of  the  eyes  and  head  is 
interpreted  as  a  skill  activity  in  both  models.  The  average  loading  of  the  model  for  a  channel 
performing  a  skilled  activity  is  computed  as  the  total  incurred  workload  divided  by  the  total  time 
needed  by  the  channel  to  perform  the  activity.  For  model  verification,  this  loading  may  be 
compared  to  the  rating  scores  reported  for  that  channel  by  the  participants  in  the  experiment. 

6.3.1. 1  Detection  Task 

Here,  we  compare  the  attention  loading  of  the  channels  for  the  model  to  that  reported  in  the 
experiment  for  the  detection  task.  We  consider  the  results  first  for  Case  I  and  then  for  Case  II. 

For  the  Case  I  model,  the  table  shows  that  the  model-derived  loading  of  3.700  for  the  visual 
channel  is  close  to  the  average  reported  value  of  3.575  and  within  the  95%  confidence  interval 
for  the  rating  mean  (4.122, 3.028).  Similarly,  the  model-derived  loading  of  3.300  for  the 
cognitive  channel  is  close  to  the  average  reported  value  of  3.367  and  within  the  95%  confidence 
interval  (5.766, 2.968).  In  addition,  the  model  derived  loading  of  2.200  for  the  psychomotor 
channel,  is  close  to  the  average  reported  value  of  1.788  and  within  the  95%  confidence  interval 
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(2.343,  1 .233).  Finally,  the  model-derived  loading  of  0.0  for  the  audio  channel  is  close  to  the 
average  reported  value  of  0.062  and  within  the  95%  confidence  interval  (0.190,  -0.066). 

For  the  Case  II  model,  the  table  shows  that  the  model  derived  loading  of  3.700  for  the  visual 
channel  is  close  to  the  average  reported  value  of  3.575  and  within  the  95%  confidence  interval 
for  the  rating  mean  (4.122,  3.028).  Similarly,  the  model-derived  loading  of  3.700  for  the 
cognitive  channel  is  close  to  the  average  reported  value  of  3.367  and  within  the  95%  confidence 
interval  (3.767, 2.968).  In  addition,  the  model-derived  loading  of  3.800  for  the  psychomotor 
channel  is  close  to  the  average  reported  value  of  3.781  and  within  the  95%  confidence  interval 
(4.329, 3.233).  Finally,  model-derived  loading  of  3.300  for  the  audio  channel  is  close  to  the 
average  reported  value  of  3.293  and  within  the  95%  confidence  interval  (3.861, 2.725).  The 
results  suggest  that  the  ratings  reported  for  the  task-allocated  attention  agree  with  those  used  in 
the  models  and  that  the  detection  task  is  closely  consistent  across  both  models  for  the  visual  and 
cognitive  channels. 

6.3. 1.2  Identification  Task 

We  compare  the  attention  loading  of  the  channels  for  the  model  to  that  reported  in  the 
experiment  for  the  identification  task.  We  consider  the  results  first  for  Case  I  and  then  for 
Case  II.  For  the  Case  I  model,  the  table  shows  that  the  model-derived  loading  of  4.000  for  the 
visual  channel  is  close  to  the  average  reported  value  of  3.973  and  within  the  95%  confidence 
interval  for  the  rating  mean  (4.462,  3.484).  Similarly,  the  model-derived  loading  of  3.700  for  the 
cognitive  channel  is  close  to  the  average  reported  value  of  3.667  and  within  the  95%  confidence 
interval  (4.146,  3.188).  In  addition,  the  model-derived  loading  of  3.400  for  the  psychomotor 
channel  (workload  of  6.80  over  2.00  seconds)  is  close  to  the  average  reported  value  of  3.039  and 
within  the  95%  confidence  interval  (3.509, 2.569).  Finally,  the  model-derived  loading  of  0.0  for 
the  audio  channel  is  equal  to  the  average  reported  value. 

For  the  Case  II  model,  the  table  shows  that  the  model-derived  loading  of  4.000  for  the  visual 
channel  is  close  to  the  average  reported  value  of  3.973  and  within  the  95%  confidence  interval 
for  the  rating  mean  (4.462,  3.484).  Similarly,  the  model-derived  loading  of  3.700  for  the 
cognitive  channel  is  close  to  the  average  reported  value  of  3.667  and  within  the  95%  confidence 
interval  (4.146, 3.188).  In  addition,  the  model-derived  loading  of  3.375  for  the  psychomotor 
channel  is  close  to  the  average  reported  value  of  3.039  and  within  the  95%  confidence  interval 
(3.509, 2.569).  Finally,  model-derived  loading  of  3.000  for  the  audio  channel  is  close  to  the 
average  reported  value  of  2.725  and  within  the  95%  confidence  interval  (3.332, 2118).  The 
workload  for  the  psychomotor  channel  is  the  sum  of  the  skilled  head  and  hand  control 
adjustments.  The  results  suggest  that  the  ratings  reported  for  the  task-allocated  attention  agree 
with  those  used  in  the  models  and  that  the  identification  task  is  fairly  consistent  across  both 
models  for  all  but  the  audio  channel. 
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6.3.2  Situational  Awareness 


Here,  the  objective  of  this  study  is  to  compare  the  ratings  of  the  SART  dimensions  to  those 
estimated  from  the  micro-activity  models,  as  determined  from  the  loading  of  the  model 
processors  during  rule-based  activity.  While  the  SART  demand  is  expressed  in  cue  properties, 
the  micro-activity  models  presented  do  not  support  the  resolution  of  the  cue  properties  for 
analysis.  However,  the  demand  sum  may  be  determined  by  the  loading  on  the  perceptual  and 
cognitive  processors  of  the  model  during  rule-based  encoding.  In  addition,  the  loading  on  the 
motor  processor  is  included  because  of  the  effect  on  result  assessment.  In  contrast,  the  micro¬ 
activity  models  presented  do  not  support  the  analysis  of  the  SART  dimensions  of  attention 
resource  capacity  (i.e.,  supply)  and  the  understanding  of  the  situation.  These  dimensions  depend 
on  properties  not  included  in  the  model.  In  particular,  the  cognitive  properties,  such  as  the  state 
of  alertness,  the  full  cognitive  capacity,  concentration,  and  spread  of  attention  focus,  are  not 
parameters  in  these  models.  The  mental  model  determining  the  understanding  is  not  part  of  the 
model  either.  For  these  reasons,  only  the  SART  demand  sum  is  compared. 

Referring  to  Table  1 1,  the  sum  of  the  workload  incurred  for  the  rule-  and  knowledge-based 
processor  activity  of  the  Case  I  model  is  33.698,  which  equates  to  an  average  loading  of  8.596 
over  3.92  seconds.  Similarly,  referring  to  Table  12,  the  workload  incurred  for  the  rule-  and 
knowledge-based  processor  activity  of  the  Case  II  model  for  the  targeting  task  and  motion 
sickness  is  168.381,  which  equates  to  an  average  loading  of  12.903  over  13.05  seconds.  These 
results  are  listed  in  Table  13  for  reference.  For  comparison  to  the  SA  ratings,  the  loading  values 
need  to  be  adjusted  for  the  difference  in  the  rating  schemes.  While  the  task-allocated  attention  is 
rated  on  a  0-to-7  scale  (with  verbal  anchors),  the  SART  dimensions  are  rated  on  a  l-to-7  scale. 
Considering  that  the  SART  demand  is  the  sum  of  three  dimensions  and  the  model  rule-  and 
knowledge-based  workload  is  summed  over  four  processors,  the  scaling  relation  between  the  two 
rating  schemes  is  y  =  0.643 *x  +3,  assuming  a  linear  relationship  between  the  equivalent  SART- 
demand  (y)  and  the  model  loading  (x). 


Table  13.  Comparison  of  SA  Demand  Sums 


SART  demand  statistics 

Model 

Treatments 

N 

Sum  mean  SD 

Workload 

Time  (sec) 

Loading 

Demand 

Ratio 

Case  I 

32 

8.225  4.279 

33.698 

3.92 

8.596 

8.527 

0.96 

Case  II 

32 

11.538  4.614 

168.381 

13.05 

12. 903 

11.294 

1.02 

With  the  scaling  relation,  the  equivalent  SART  demand  for  the  Case  I  model  is  8.527;  this  value 
is  within  the  95%  confidence  interval  (9.770, 6.680)  for  the  average  demand  sum  of  8.225 
reported  by  the  participants  for  experiment,  according  to  the  statistics  listed  in  Table  6.  Here,  the 
ratio  of  the  demand  sum  to  the  model  demand  is  0.96.  Similarly,  the  equivalent  SART  demand 
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for  the  Case  II  model  is  1 1 .294,  which  is  within  the  95%  confidence  interval  (13.204, 9.872)  for 
the  average  demand  sum  of  1 1 .538  reported  by  the  participants.  The  ratio  of  the  demand  sum  to 
the  model  demand  is  1.02.  These  results  suggest  that  the  SART  demand  sum  is  measured  by  the 
equivalent  demand  for  the  model  loading. 

6.3.3  Motion  Sickness 

Motion  sickness  is  interpreted  as  being  caused  by  the  sensory  conflicts  in  the  somatic  perceptual 
processor  and  the  associated  cognitive  assessment.  An  experienced  gunner  is  assumed  to 
naturally  move  his  head  in  such  a  way  as  to  induce  movement  of  the  vestibular  system  to  counter 
conflicts  with  the  visual  scene.  However,  since  the  activity  is  pipelined,  the  motor  activity  is  in 
response  to  a  previous  visual  image  instead  of  the  most  recent.  The  result  is  a  skill-based 
sequence  of  somatic  perception  followed  by  a  countering  head  shift  that  is  generated  with  each 
visual  glimpse.  The  conflict  remaining  is  resolved  by  a  rule-based  cognitive  activity.  Since  our 
model  does  not  have  the  resolution  to  differentiate  between  symptom  sources,  we  compare  the 
model  to  the  total  severity  of  motion  sickness.  Furthermore,  since  the  motion  sickness  reported 
was  slight  for  the  Case  I  model,  the  symptoms  are  ignored  in  the  time  line  of  Table  11. 
Considering  the  somatic  perception,  the  associated  head  movement,  and  the  resulting  cognitive 
assessment,  along  with  the  associated  interferences  in  Table  12  for  the  Case  II  model,  the  total 
severity  loading  is  7.669,  since  the  incurred  workload  is  42.793  over  the  model  time  of 
5.58  seconds.  For  comparison  to  the  motion  sickness  ratings,  the  loading  values  need  to  be 
adjusted  for  the  difference  in  the  rating  schemes.  Although  the  task-allocated  attention  is  rated  on 
a  0-to-7  scale,  each  of  the  motion  sickness  dimensions  is  rated  on  a  0-to-3  scale.  Although  the 
model  workload  is  summed  over  the  three  processors  for  a  0-to-2 1  scale,  the  total  severity  is  a 
weighted  sum  of  the  16  motion  sickness  dimensions  resulting  in  a  0-to- 120.54  scale  (Kennedy, 
Lane,  Lilienthal,  Berbaum,  &  Hettinger,  1992).  However,  the  severity  distribution  for  the  Case  II 
treatments  is  heavily  skewed  toward  lower  values  (skewness  =  1.510,  N  =  16,  standard 
error  =  0.564),  and  weighting  the  scale  by  the  ratio  of  the  median  (14.480)  to  the  mean  (22.199) 
results  in  a  0-to-78.62  scale.  The  scaling  relation  between  the  two  rating  schemes  is  y  =  3.744*x, 
assuming  a  linear  relationship  between  the  equivalent  total  severity  (y)  and  the  model  loading 
(x).  With  this  relation,  the  equivalent  total  severity  for  the  Case  II  model  is  28.713,  a  value 
within  the  95%  confidence  interval  (34.634, 9.764)  for  the  average  total  severity  of  22.199 
(N  =  16,  standard  deviation  =  23.342)  reported  by  the  participants. 

6.4  Relation  to  Perceived  Workload 

We  compare  the  workload  incurred  for  the  micro-activity  time  line  models  to  the  perceived 
workload  reported  by  the  experimental  participants.  We  first  compare  the  total  perceived 
workload  to  the  total  workload  incurred  by  the  micro-activity  models.  We  then  compare  the 
workload  reported  for  each  of  the  perceived  dimensions.  Essential  to  this  work  is  the  designation 
of  sources  within  the  model  of  the  workload  for  the  dimensions. 
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6.4.1  Total  Workload 

The  NASA-TLX  of  mental  workload  is  interpreted  as  the  perceived  task  loading  over  the 
duration  of  the  task.  As  described  before,  the  total  perceived  TLX  loading  is  the  sum  of  the 
loadings  for  the  measure  dimensions.  With  this  interpretation,  the  workload  incurred  is  the 
product  of  the  task  loading  and  the  course  time.  Referring  to  Table  1 1,  the  sum  of  the  total 
workload  incurred  for  the  Case  I  model  is  60.464,  which  equates  to  an  average  loading  of  8.1 16 
over  the  7.45  seconds.  Similarly,  referring  to  Table  12,  the  total  workload  incurred  for  the  Case 
II  model  is  245.1 14,  which  equates  to  an  average  loading  of  15.886  over  the  15.43  seconds. 
These  results  are  listed  in  Table  14  for  reference. 

For  comparison  to  the  total  TLX  ratings,  the  loading  values  need  to  be  adjusted  for  the  difference 
in  the  rating  schemes.  Although  the  task-allocated  attention  is  rated  on  a  0-to-7  scale,  the  TLX 
dimensions  are  rated  on  a  0-to-9  scale.  Considering  that  the  total  TLX  is  the  sum  of  six 
dimensions  and  the  Case  I  model  workload  is  summed  over  four  processors,  the  scaling  relation 
between  the  two  rating  schemes  is  y  =  1.928*x,  assuming  a  linear  relationship  between  the 
equivalent  total  TLX-rating  (y)  and  the  model  loading  (x).  With  the  scaling  relation,  the 
equivalent  total  TLX  for  the  Case  I  model  is  15.647;  this  value  is  within  the  95%  confidence 
interval  (10.367, 17.298)  for  the  average  total  TLX  rating  of  13.966  reported  by  the  participants, 
according  to  the  statistics  in  Table  6.  Here,  the  ratio  of  the  total  TLX  loading  to  the  equivalent 
model  loading  is  0.90.  Since  the  Case  II  model  workload  is  summed  over  five  processors,  the 
scaling  relation  between  the  two  rating  schemes  is  y  =  1.543*x.  Consequently,  the  equivalent 
total  TLX  for  the  Case  II  model  is  24.51 1,  which  is  within  the  95%  confidence  interval  (32.075, 
24. 1 25)  for  the  average  sum  of  28. 1 00  reported  by  the  participants.  The  ratio  of  the  total  TLX 
loading  to  the  model  equivalent  is  1.14.  These  results  suggest  that  the  total  TLX  loading  is 
measured  by  the  equivalent  model  loading. 


Table  14.  Comparison  of  TLX  Total  Workload  to  Model  Total  Loading 


TLX  statistics 

Model 

Treatments 

N 

Sum  mean 

SD 

Workload 

Time  (sec) 

Loading 

Demand 

Ratio 

Case  I 

32 

13.966 

9.958 

60.464 

7.45 

8.116 

15.647 

0.90 

Case  II 

16 

11.011 

245.114 

15.43 

15.886 

24.511 

1.14 

6.4.2  Workload  Dimensions 

We  consider  the  relation  of  the  workload  dimensions  to  the  micro-activity  models.  In  particular, 
micro-processor  configurations  are  mapped  to  the  dimensions,  and  the  resulting  micro-activity 
loading  values  are  compared  to  the  rating  statistics  when  possible. 
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6.4.2. 1  Mental  Demand 

The  dimension  rates  the  amount  of  mental  and  perceptual  activity  that  is  needed  to  perform  the 
task.  We  interpret  mental  demand  as  the  amount  of  perceptual  and  cognitive  activity  used  at  the 
skill  and  rule  levels  but  not  knowledge  recall,  which  is  separate  from  the  somatic  response. 
Referring  to  Table  1 1,  the  sum  of  the  workload  incurred  for  the  Case  I  model  is  33.897,  which 
equates  to  an  average  loading  of  5.575  over  6.08  seconds.  Similarly,  referring  to  Table  12,  the 
workload  incurred  for  the  Case  II  model  is  148.292,  which  equates  to  an  average  loading  of 
12.493  over  1 1.87  seconds.  For  comparison  to  the  TLX  mental  demand  ratings,  the  loading 
values  need  to  be  adjusted  for  the  difference  in  the  rating  schemes.  Although  the  task-allocated 
attention  is  rated  on  a  0-to-7  scale,  the  TLX  dimension  is  rated  on  a  0-to-9  scale.  Since  the  Case  I 
model  workload  is  summed  over  three  processors  (audio,  visual,  and  cognitive),  the  scaling 
relation  between  the  two  rating  schemes  is  y  =  0.4286*x,  assuming  a  linear  relationship  between 
the  mental  demand  rating  (y)  and  the  model  loading  (x).  With  the  scaling  relation,  the  equivalent 
mental  demand  is  2.389  for  Case  I.  This  value  is  within  the  95%  confidence  interval  (3.309, 
1.503)  for  the  average  mental  demand  rating  of  2.406  reported  by  the  participants  (N  =  32, 
standard  deviation  =  2.503)  for  the  direct  vision  treatments.  Similarly,  the  Case  II  model  is 
summed  over  the  same  three  processors,  and  the  equivalent  mental  demand  is  5.354  for  the  Case 
II  model,  which  is  within  the  95%  confidence  interval  (5.569, 3.612)  for  the  average  mental 
demand  rating  of  4.591  reported  by  the  participants  (N  =  32,  standard  deviation  =  2.710)  for  the 
indirect  vision  treatments  with  the  HMD. 

6.4.2.2  Physical  Demand 

The  dimension  rates  the  amount  of  physical  activity  required  to  perform  the  task.  We  interpret 
physical  demand  as  the  amount  of  perceptual  and  motor  skill-based  activity  used,  which  is 
separate  from  the  somatic  response.  Referring  to  Table  1 1,  the  sum  of  the  workload  incurred  for 
the  Case  I  model  is  19.700,  which  equates  to  an  average  loading  of  5.472  over  3.60  seconds. 
With  the  scaling  relation  for  the  three-processor  configuration  (audio,  visual,  and  motor),  the 
equivalent  physical  demand  is  2.345  for  Case  I.  This  value  is  outside  the  95%  confidence 
interval  (1.745, 0.698)  for  the  average  physical  demand  rating  of  1.222  reported  by  the 
participants  (N=32,  standard  deviation  =  1.450)  for  the  direct  vision  treatments.  Similarly, 
referring  to  Table  12,  the  workload  incurred  for  the  Case  II  model  is  49.076,  which  equates  to  an 
average  loading  of  4.461  over  1 1.00  seconds.  Considering  that  the  model  workload  is  summed 
over  four  processors,  the  scaling  relation  between  the  two  rating  schemes  is  y  =  0.32 14*x, 
assuming  a  linear  relationship  between  the  physical  demand  rating  (y)  and  the  model  loading  (x). 
Similarly,  the  equivalent  physical  demand  is  1.434  for  the  Case  II  model,  which  is  close  to  the 
95%  confidence  interval  (3.416, 1.708)  for  the  average  physical  demand  rating  of  2.562  reported 
by  the  participants  (N  =  32,  standard  deviation  =  2.366)  for  the  indirect  vision  treatments  with 
the  HMD. 
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6.4.2.3  Temporal  Demand 

The  dimension  rates  the  pace  at  which  the  task  was  performed.  We  interpret  temporal  demand  as 
the  amount  of  perceptual  activity  for  both  the  task  and  the  somatic  response.  Referring  to  Table 
1 1,  the  sum  of  the  workload  incurred  for  the  Case  I  model  is  12.750,  which  equates  to  an  average 
loading  of  3.984  over  3.20  seconds.  With  the  scaling  relation  for  two  processors  (audio  and 
visual),  given  by  y  =  0.6428 *x,  the  equivalent  temporal  demand  is  2.561,  which  is  within  the 
95%  confidence  interval  (4.157,  2.212)  for  the  average  temporal  demand  rating  of  3.184  reported 
for  that  treatment  (N=32,  standard  deviation  =  2.694).  Similarly,  referring  to  Table  12,  the 
workload  incurred  for  the  Case  II  model  is  1 18.888,  which  equates  to  an  average  loading  of 
14.769  over  8.05  seconds.  Again,  using  a  scaling  relation  of  0.4285  for  three  processors,  the 
equivalent  temporal  demand  is  6.328,  which  is  near  the  95%  confidence  interval  (5.696,  3.835) 
for  the  average  temporal  demand  rating  of 4.766  reported  for  that  treatment  (N  =  32,  standard 
deviation  =  2.577). 

Analyses  are  not  made  for  the  TLX  interaction  dimensions  of  effort,  performance,  and 
frustration,  since  the  micro-activity  models  do  not  include  appropriate  mental  modeling  as 
components.  The  rating  of  the  effort  dimension  depends  on  previous  experience  with  the  tasks.  A 
participant  with  more  experience  requires  less  effort  to  perform  the  same  task.  The  performance 
dimension  rates  the  success  of  the  participant  in  accomplishing  the  task  goals.  The  frustration 
dimension  rates  the  satisfaction  with  performance.  The  evaluation  of  these  dimensions  requires 
inclusion  of  mental  models  of  experience,  goals,  and  motivation,  which  are  not  part  of  the  micro¬ 
activity  models  developed  here.  In  the  next  section,  the  results  of  the  mental  workload  analysis 
are  incorporated  into  a  task  analysis  workload  simulation  of  the  targeting  task.  The  results  are 
that  the  task  workload  is  expanded  to  include  additional  components  of  perceived  workload  in 
the  simulation  through  the  effects  of  SA  and  motion  sickness. 


7.  Mental  Workload  Simulation 


Following  the  micro-activity  models  described  before,  the  mental  workload  is  incorporated  into  a 
task  analysis  workload  simulation.  In  this  development,  workload  is  defined  as  the  amount  of 
effort,  both  physical  and  psychological,  that  is  expended  in  response  to  system  demands 
according  to  an  internal  standard  of  performance  (Stein,  1993).  Here,  target  detection  and 
identification  are  self-paced  tasks  and  the  task  load  is  the  time  of  execution  and  the  error  rate 
incurred  according  to  the  standards  established  during  training  (Smyth,  2002b).  In  this  report,  the 
task  load  is  described  in  the  section  about  task  performance,  while  the  mental  workload  is 
described  in  the  sections  about  mental  workload  measures  and  micro-activity  time  line. 
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7.1  Simulation  Modeling  Program 

The  Improved  Performance  Research  Integration  Tool  (IMPRINT)  simulation  modeling  program 
(Allender,  1998),  which  is  used  to  model  the  workload  of  the  targeting  task,  was  developed  for 
ARL  by  Micro- Analysis  &  Design,  Inc.  The  program  consists  of  a  set  of  Windows-based 
automated  aids  to  assist  analysts  in  conducting  human  performance  analyses.  In  addition  to  being 
useful  for  estimating  crew  workload,  IMPRINT  provides  the  means  for  estimating  manpower, 
personnel,  and  training  requirements  and  constraints  for  new  weapon  systems  early  in  the 
acquisition  process. 

An  essential  step  in  the  workload  analysis  is  defining  the  system  mission  and  the  functions  and 
tasks  to  be  performed  by  the  crew  in  that  mission.  A  hierarchy  of  task  analysis  network  diagrams 
is  used  to  depict  the  functions  and  the  tasks.  The  network  links  together  the  tasks  in  a  sequence 
of  activities  with  the  tasks  denoted  by  the  network  branches  and  task  completion  by  the  nodes. 
Depending  on  the  result  of  a  task  activity,  branching  to  the  next  task  can  occur  among  tasks 
connected  to  the  same  node.  This  branching  is  dictated  by  a  set  of  branching  rules,  which  may  be 
probabilistic,  repeating,  or  tactical  in  form.  Associated  with  each  task  are  the  specifications  for 
the  statistics  of  the  performance  time,  the  accuracy  of  the  result,  and  the  effects  on  the  branching 
rules.  In  an  advanced  version,  the  task  may  be  subdivided  into  micro-state  activities  with 
associated  component  times.  Also  associated  with  each  task  are  attention  loading  values  for 
activation  of  the  human  visual,  auditory,  cognitive,  and  psychomotor  information  processing 
channels  during  task  performance  and  the  additional  loading  attributable  to  channel  conflicts. 

The  attention  loading  values  may  be  combined  into  an  overall  workload  measure.  Workload 
management  strategies  may  be  designated  to  handle  work  overload.  The  machine  displays  and 
controls  with  which  the  crew  member  interfaces  are  assigned  channels  as  resources  that  are 
activated  for  operations.  In  turn,  the  interfaces  are  assigned  to  the  tasks.  Finally,  the  individual 
crew  members  are  assigned  tasks. 

When  the  simulation  model  is  executed,  the  tasks  are  scheduled  and  performed  as  specified  in 
the  flow  diagrams  and  branching  logic.  As  the  tasks  are  performed,  the  total  instantaneous 
workload  prediction  is  calculated  as  the  weighted  sum  of  the  load  on  each  of  the  channels  at  a 
moment  in  time  as  rated  with  the  loading  values  and  a  factor  that  accounts  for  the  amount  of 
conflict  between  and  within  resources  that  are  used  in  parallel.  The  predicted  workload  output  is 
a  measure  of  the  amount  of  effort  that  the  tasks,  the  scenario,  and  the  interfaces  are  imposing  on 
the  simulated  operator. 

This  is,  of  course,  a  limited  presentation  of  the  full  capabilities  of  the  simulation  program  as  it 
applies  to  workload  for  the  extremely  simplistic  task  of  targeting.  This  is  especially  true  when 
compared  to  the  full  range  of  successful  application  to  simulating  in-depth  operations  and 
maintaining  a  wide  range  of  military  vehicles  and  helicopters.  However,  the  construction  of  a 
simulation  model  allows  us  to  consider  perceived  workload  as  an  addition  to  the  task  workload, 
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particularly  the  effects  of  SA  and  motion  sickness  that  result  from  our  field  study  and 
consequential  analysis. 

7.2  Simulation  Model 

Next,  we  construct  a  simulation  model  for  the  targeting  task.  We  consider  the  task  network 
diagrams,  the  resource  interface  assignments,  channel  loading  values,  task  activity  times, 
and  branching  decision  rules,  and  we  discuss  a  representative  simulation  output. 

7.2.1  Task  Network  Diagrams 

Figure  6  is  a  function-level  network  flow  diagram  for  the  IMPRINT  representation  of  the  task. 
The  figure  shows  an  experimental  trial  consisting  of  a  trial  start,  target  detection,  identification, 
correct  identification,  and  trial  end.  The  figure  includes  IMPRINT  decision  nodes  for  the 
probabilities  of  detection,  identification,  and  correct  identification.  The  tasks  for  the  detection 
and  identification  stages  are  further  described  in  the  following  network  flow  diagrams. 


Figure  6.  Functional  Flow  Diagram. 


7.2.1. 1  Detection  Task 

Figure  7  is  an  IMPRINT  task  level  flow  diagram  for  the  detection  task.  Following  the  section 
about  task  analysis  (see  Figure  1),  the  diagram  shows  the  task  modeled  as  a  closed  loop  sequence 
of  visual  glimpses  executed  with  eye  and  head  shifts  for  indexing  the  visual  lobe.  Vehicular 
motion  causes  the  participant  to  re-center  the  view  for  each  glimpse.  The  sequencing  among  the 
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loop  branches  is  determined  by  IMPRINT  decision  rules.  Here,  the  decision  rule  for  the  node  T1 
determines  the  number  of  times  that  the  vision  system  must  be  indexed  to  view  the  target.  For  the 
return  loop,  the  figure  shows  two  path  segments,  one  for  indexing  the  eyes  and  the  other  for 
indexing  the  head  when  the  vision  is  obscured  by  the  viewing  FOV,  as  determined  by  the 
decision  rule  for  node  T2.  Once  the  visual  lobe  is  directed  to  the  target  location,  the  decision  rule 
for  node  T1  passes  the  simulation  to  the  voice  response  and  the  task  end  node.  Since  in  practice, 
an  IMPRINT  task  flow  is  commonly  diagrammed  in  terms  of  observable  measures,  these 
activities  may  be  summarized  as  target  search,  and  the  detection  task  diagrammed  as  command 
response,  search,  and  verbal  response  following  detection. 

7.2. 1.2  Identification  Task 

Figure  8  is  an  IMPRINT  task  flow  diagram  for  the  identification  task.  Following  the  task 
analysis  (see  Figure  2),  the  diagram  shows  the  task  as  a  closed  loop  sequence  of  zoom  (or 
magnification)  adjustments  in  the  viewing  power,  followed  by  a  visual  glimpse  at  the  target. 
Zoom  adjustment  and  vehicular  motion  cause  the  participant  to  re-center  the  view  for  each 
glimpse.  The  IMPRINT  decision  rule  for  the  node  T3  determines  the  number  of  times  that  the 
adjustments  must  be  made  to  reach  the  proper  magnification  to  identify  the  marker  on  the  target. 
Once  the  adjustments  are  completed,  the  decision  rule  passes  the  identification  to  the  voice 
response  and  the  task  end  node.  Again,  these  activities  may  be  summarized  as  zooming,  and  the 
identification  task  may  be  diagrammed  as  control  placement,  zoom,  and  verbal  response 
following  identification. 


Figure  7.  Task-Level  Flow  Diagram  for  Detection. 
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Figure  8.  Task-Level  Flow  Diagram  for  Identification. 


7.2.2  Resource  Interface  Channels 

Table  15  lists  the  resources  and  interfaces  for  the  IMPRINT  task  model.  As  can  be  seen  from  the 
table,  the  interfaces  between  the  participant  and  vehicle  are  the  visual  display,  the  auditory 
display,  and  the  manual  controls.  The  attentional  resources  of  the  participant  include  the  visual, 
auditory,  cognitive,  and  motor  processors.  The  speech  interaction  with  the  experimenter  is  not 
included  in  the  analysis. 

Table  15.  Workload  Attention  Resources  and  Interface  Channels 


Resources 

Interfaces 

Visual 

Visual  display 

Auditory 

Earphones 

Cognitive 

Visual  display 

Motor 

Manual  controls 

Table  16  shows  the  relations  between  the  interfaces  and  resources,  that  is,  which  interfaces  place 
demands  upon  the  attentional  resources  in  the  IMPRINT  model.  The  table  is  a  simple  mapping  of 
the  input  from  the  vehicle  displays  to  the  visual,  audio,  and  cognitive  channels,  with  the  manual 
controls  as  the  output  of  the  motor  channel. 
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Table  16.  Relation  of  Operator  Interface  Channels  to  Attention  Resources 


Attention  Resources 

Interface 

visual 

auditory 

cognitive 

motor 

Visual  Display 
Earphones 

Manual  controls 

X 

X 

X 

X 

7.2.3  Channel  Loading  Values 

Tables  17  and  18  list  workload  demands  placed  by  the  tasks  upon  the  resources  via  the  interface 
channels  in  the  IMPRINT  model,  for  the  two  treatments  of  Case  I:  direct  vision,  stationary, 
without  sound  localization,  and  Case  II:  HMD,  moving  vehicle,  with  sound  localization, 
respectively.  In  the  tables,  the  loading  values  for  the  tasks  are  predicted  from  the  micro-activity 
models  of  Tables  1 1  and  12.  In  this  process,  the  loading  for  the  task  is  calculated  as  the  ratio  of 
incurred  workload  during  the  task  to  the  time  that  the  loading  occurred.  In  addition  to  listing  the 
skill-based  task  allocation  and  rule-based  SA  demand  for  the  tasks,  the  tables  list  the  motion 
sickness  severity.  The  total  perceived  work  loading  is  also  listed.  In  the  micro-modeling  section, 
it  was  demonstrated  that  the  experimental  measures  agreed  with  the  loading  values  derived  from 
appropriate  processor  configurations  for  the  micro-model  time  lines  following  dimension 
scaling.  In  this  maimer,  the  measures  are  scaled  to  the  7-point  (0  to  7)  loading  factors  scale 
(McCracken  &  Aldrich,  1984),  used  for  allocated  attention  in  the  IMPRINT  simulation 
modeling. 


Table  17.  Average  Attention  Loading  Values  for  Case  I:  Direct  Vision,  Stationary,  Without  Sound 


Task 

Visual 

Task  Allocation 
Cognitive  Audio 

Motor 

Total 

Awareness 

Demand 

Motion 

Sickness 

Total 

Workload 

(a)  Detection- 

Command 

0.00 

0.00 

0.00 

0.00 

0.00 

5.31 

0.00 

5.31 

Search 

3.70 

3.30 

0.00 

2.20 

8.21 

0.00 

0.00 

8.21 

Voice  Response 

3.70 

0.00 

0.00 

0.00 

3.70 

8.19 

0.00 

7.89 

(b)  Identification- 

Place  Controls 

4.00 

3.70 

0.00 

0.00 

6.59 

3.76 

0.00 

4.48 

Zoom 

4.00 

3.70 

0.00 

4.00 

9.99 

0.00 

0.00 

9.99 

Voice  Response 

0.00 

0.00 

0.00 

0.00 

0.00 

7.95 

0.00 

7.95 
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Table  18.  Average  Attention  Loading  Values  for  Case  II:  Indirect  Vision,  Moving,  With  Sound 
Localization 


Task 

Visual 

Task  Allocation 
Cognitive  Audio 

Motor 

Total 

Awareness 

Demand 

Motion 

Sickness 

Total 

Workload 

(a)  Detection- 

Command 

0.00 

0.00 

3.30 

0.00 

3.30 

6.80 

0.00 

7.50 

Search 

3.70 

3.70 

3.30 

3.80 

5.43 

14.51 

7.66 

16.31 

Voice  Response 

4.00 

0.00 

3.30 

0.00 

3.87 

17.62 

9.86 

19.59 

(b)  Identification- 

Place  Controls 

4.00 

3.70 

3.00 

0.00 

3.75 

11.18 

8.82 

12.87 

Zoom 

4.00 

3.70 

3.00 

3.37 

8.77 

16.59 

7.84 

17.86 

Voice  Response 

0.00 

0.00 

3.00 

0.00 

3.00 

11.84 

10.49 

12.75 

In  each  case,  the  table  shows  the  loading  applied  over  the  full  sub-task.  These  sub-tasks  are  the 
command  response,  search,  and  verbal  response  for  detection,  placement  of  controls,  zoom,  and 
verbal  response  for  identification.  This  is  true  despite  the  fact  that  the  activities  of  glimpse,  index 
eyes,  and  re-center  in  the  detection  search  task  are  sequential  for  both  cases  according  to 
Tables  1 1  and  12.  Further,  the  identification  zoom  glimpse  activity  is  concurrent  with  the 
activities  of  the  head  and  hand  adjustments  for  Case  I  but  sequential  for  Case  II.  In  concept,  the 
loading  specifies  the  attention  demand  that  is  required  to  perform  the  mental  function  during  the 
time  that  it  is  activated  as  part  of  the  task.  As  demonstrated  in  the  micro-activity  models,  a 
mental  function  is  activated  during  part  of  the  task  and  the  loading  only  applies  to  that  period  and 
not  to  the  entire  task.  However,  we  may  compute  the  workload  incurred  as  the  loading  applied 
over  the  task  duration  and  summed  over  tasks;  this  approach  results  in  workloads  that  are  close 
in  value  via  the  model  loading  values  or  those  that  were  experimentally  collected.  The 
implication  is  that  we  may  consider  the  loading  values  as  applying  to  the  full  sub-task  duration. 
Note  that  in  this  simulation,  the  total  loading  is  computed  as  a  weighted  sum  of  the  loading 
values  on  the  processors,  in  which  the  weights  are  the  ratio  of  the  duration  of  the  processor 
activity  to  the  behavior  duration  for  the  total  loading  as  determined  from  the  micro-activity 
model. 

7.2.4  Task  Activity  Times 

Table  19  lists  the  task  activity  times  used  in  the  simulation  for  the  two  treatment  cases.  The  times 
are  from  the  micro-activity  models  of  Tables  1 1  and  12  with  the  processor  activities  blocked  to 
match  the  task-level  flow  diagrams  of  Figures  7  and  8.  The  blocking  used  corresponds  to  the 
sub-tasks  of  command  response,  search,  and  verbal  response  for  detection,  placement  of  controls, 
zoom,  and  verbal  response  for  identification. 
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Table  19.  Task  Activity  Times  (seconds) 


Task 

Experimental  Treatment 

Case  I 

Case  II 

(a)  Detection- 

Command  Response 

0.17 

0.47 

Search 

1.40 

6.70 

Voice  Response 

1.52 

1.60 

Total 

3.09 

8.77 

(b)  Identification- 

Place  Controls 

0.76 

1.38 

Zoom 

2.00 

3.96 

Voice  Response 

1.25 

1.32 

Total 

4.01 

6.66 

Grand  Total 

7.10 

15.43 

7.2.5  Branching  Decision  Rules 

Table  20  lists  the  scenario  variables  used  in  the  simulation;  the  status  of  the  variables  is  used  to 
control  the  branching  logic  of  the  task  flow.  Table  21  lists  the  IMPRINT  branching  logic  for  the 
probability  nodes  of  Figure  6.  Tables  22  and  23  list  the  decision  rules  for  the  looping  nodes  of 
Figure  7  for  the  detection  task.  Table  24  lists  the  rules  for  the  looping  node  of  Figure  8  for  the 
identification  task.  The  probabilities  listed  in  Table  21  are  derived  from  the  experimental  results. 
In  the  function  diagram  of  Figure  6,  the  probability  nodes  control  the  branching  of  the  task  flow. 
The  task  times  used  to  compute  the  logic  of  the  decision  nodes  in  Tables  22  through  24  are  listed 
in  Table  19.  The  task  loop  cycle  times  are  computed  from  the  sum  of  the  times  for  the  micro¬ 
activities  needed  to  complete  the  loop  cycle,  as  taken  from  Tables  1 1  and  12.  These  results  agree 
with  the  times  reported  in  the  Micromodel  library  of  IMPRINT  (Allender,  1998;  see  also  Little  et 
al.,  1993).  The  loop  cycle  is  composed  of  a  sequence  of  micro-level  activities  executed  by  the 
human  to  acquire  and  process  information,  make  decisions,  and  execute  actions.  In  the  task 
diagrams  of  Figures  7  and  8,  task  choices  are  decided  at  looping  nodes  as  a  function  of  the  state 
of  the  scenario  variables. 

7.3  Simulation  Results 

Figure  9  is  a  representative  plot  of  the  predicted  mental  loading  produced  by  a  simulation  of  the 
Case  II  treatment  of  HMD,  moving  vehicle,  and  sound  localization.  The  plot  shows  the  loading 
values  from  Table  18  for  the  total  task  allocation,  SA  demand,  motion  sickness  severity,  and 
perceived  total  workload,  as  a  function  of  the  task  activities.  The  plot  demonstrates  that  proper 
mapping  of  the  elements  of  the  simulation  task  flow  network  with  the  clusters  of  the  mental 
workload  measures  can  result  in  a  reasonable  simulation  of  the  mental  workload  that  occurs 
during  the  targeting  tasks. 
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Table  20.  Scenario  Variables 


Variable  name 

Type 

Role 

time  det 

real 

time  to  detect  target  from  start  of  trial 

time  id 

real 

time  to  identify  target  from  time  of  detection 

cycle  det 

real 

time  period  between  indexing  visual  lobe 

cycle  id 

real 

time  period  between  indexing  zoom  control 

tgt  ang 

real 

angular  offset  of  target  from  initial  search  orientation 

fov  device 

real 

FOV  of  viewing  device 

fov  glimpse 

real 

effective  FOV  of  glimpse  viewing  lobe 

max  vis 

integer 

total  number  of  visual  lobe  shifts  to  reach  target 

index  vis 

index  of  visual  lobe  shifts 

index  head 

;;;; 

index  of  number  of  head  shifts  for  vision  device 

max  eyes 

total  number  of  eye  shifts  to  span  vision  device 

index  eyes 

index  of  number  of  eye  shifts  for  head  position 

max  zoom 

total  number  of  manual  zoom  shifts 

index_zoom 

index  of  manual  zoom  shifts 

Table  21.  Branching  Logic  for  Probability  Nodes  of  Trial  Functional  Flow  Diagram  in  Figure  6 

1.  Probability  Node,  PI:  Determine  if  successful  target  detection. 

Case  I  treatment:  Direct  viewing,  stationary,  no  sound  localization 

Following  node 

logic 

Detection 

p  = 

0.901; 

End  Trial 

p  = 

0.099; 

Case  II  Treatment:  HMD,  moving  vehicle,  sound  localization- 

Following  node 

logic 

Detection 

p  = 

0.307; 

End  Trial 

p  = 

0.693; 

II.  Probability  Node,  P2:  Determine  if  successful  target  identification. 

Case  I  treatment:  Direct  viewing,  stationary,  no  sound  localization- 

Following  node 

logic 

Identification 

P  = 

0.948; 

End  Trial 

p  = 

0.052; 

Case  II  Treatment:  HMD,  moving  vehicle,  sound  localization- 
Following  node  logic 

Identification  P  =  0.305; 

End  Trial  P  =  0.695; 

III.  Probability  Node,  P3:  Determine  if  correct  identification 

Case  I  treatment:  Direct  viewing,  stationary,  no  sound  localization- 
Following  node  logic 

Correctly  Identified  P  -  0.872; 

End  Trial  P  =  0.128; 

Case  II  Treatment:  HMD,  moving  vehicle,  sound  localization- 
Following  node  logic 

Correctly  Identified  P  -  0.611; 

End  Trial _ P  =  0.389: _ 
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Table  22.  Branching  Logic  for  Decision  Node  T1  of  Detection  Task  Flow  Diagram  (see  Figure  7) 


Purpose:  Determine  when  target  detected 
Method:  Index  visual  lobe  to  target  position. 

(a)  Initialize  variables: 

Task:  Detection 
Sub-task:  Command  Response 

Effect:  Beginning 

Case  I  Treatment:  direct  vision,  stationary,  no  sound  cues 

Variable  name  value  units 

timedet  1.400  seconds 

cycle_det  200  millisecs 

Case  II  Treatment:  HMD,  moving,  sound  localization 

Variable  name  value  units 

time_det  6.700  seconds 

cyclejiet  418  millisecs 

Effect:  Ending 

Algorithm:  max  vis  :=  time_det/cycle_det; 

Variable  name  value  units 

index_vis  0  count 

(b)  Increment  logic: 

Task:  Detection 
Sub-task:  Glimpse 
Effect:  Ending 

Algorithm:  index  vis  +=  1; 

(c)  Branching  logic: 

Following  node  logic 

Recenter  View  index  vis  <  max_vis; 

Voice  Response  index_vis  >=  max_vis; 
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Table  23.  Branching  Logic  for  Decision  Node  T2  of  Detection  Task  (see  Figure  7) 


Purpose:  Control  head  movement. 

Method:  Count  number  of  eye  shifts  to  reach  head  movement. 

(a)  Initialize  variables 

Task:  Detection 

Sub-task:  Command  Response 

Effect:  beginning 

Case  I  Treatment:  direct  vision,  stationary,  no  sound  cues 

Variable  name  value  units 

tgt_ang  30.0  degrees 

fov_device  85.0  degrees 

Case  II  Treatment:  HMD,  moving,  sound  localization 

Variable  name  value  units 

tgt_ang  30.0  degrees 

fov_device  13.125  degrees 


Effect:  Ending 

Algorithm:  fov^glimpse  :=  tgt_ang/max_vis; 

max_eyes  :=  fov_device/fov_glimpse; 

Variable  name  value  units 

indexeyes  0  count 

index Jiead  0  count 

(b)  Increment  logic: 

Task:  Detection 
Sub-task:  Index  Eyes 
Effect:  Ending 
Algorithm:  index_eyes+=  1; 

Task:  Detection 
Sub-task:  Index  Head 
Effect:  Ending 
Algorithm:  index  Jiead  +=  1; 
index_eyes  :=  0; 

(c)  Branching  logic: 

Following  node  logic 

Index  Eyes  index  eyes  <max_eyes; 

Index  Head  index_eyes  >=  max_eyes; 
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Table  24.  Branching  Logic  for  Decision  Node  T3  of  Identification  Task  Flow  Diagram  (see  Figure  8) 


Purpose:  Control  zoom  movements. 

Method:  Count  zoom  indexes  to  see  Landolt  ring  gap. 

(a)  Initialize  variables: 

Task:  Identification 
Sub-task:  Place  Zoom  Controls 

Effect:  beginning 

Case  I  Treatment:  direct  vision,  stationary,  no  sound  cues 

Variable  name  value  units 

timejd  2.00  seconds 

cycle jd  100  millisecs 

Case  II  Treatment:  HMD,  moving,  sound  localization 

Variable  name  value  units 

time_id  3.96  seconds 

cyclejd  440  millisecs . 


Effect:  Ending 

Algorithm:  max_zoom  :=  timejd/cyclejd; 

Variable  name  value  units 

indexzoom  0  count 

(b)  Increment  logic: 

Task:  Identification 
Sub-task:  Glimpse 
Effect:  Ending 
Algorithm:  index_zoom  +=  1; 

(c)  Branching  logic: 

Following  node  logic 

Re-center  View  index_zoom  <max_zoom; 

Voice  Response  index_zoom  >=  maxzoom; 
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Figure  9.  Simulation  Output  for  Case  II:  HMD,  Moving  Vehicle,  With  Sound  Localization. 


7.4  Workload  Cost  Considerations 

Human  factors  analysts  may  use  the  simulation  of  task  workload  to  predict  the  effects  on 
performance.  In  a  two-person  vehicle,  the  gunner  may  be  expected  to  perform  other  functions 
such  as  navigation  and  communications.  An  increase  in  task  workload,  the  demand  on  SA,  and 
the  chance  of  motion  sickness  may  increase  the  psychological  stress  on  the  gunner.  At  high  stress 
levels,  the  resulting  physiological  arousal  is  detrimental  to  performance.  The  immediate  effect  is 
to  produce  tunneling  or  narrowing  of  the  perceptual  and  cognitive  attention,  in  order  to 
concentrate  on  the  detection  and  identification  tasks  while  ignoring  the  surroundings.  There  is  a 
tendency  toward  working  memory  loss  with  reduced  capacity  for  processing  new  or  complex 
material  and  a  focus  on  well-learned  responses  and  a  rapid  execution  of  these  responses  at  the 
expense  of  possible  errors.  The  resultant  shedding  of  secondary  tasks  may  not  be  done  in  an 
optimal  order  for  successfully  coping  with  the  problem  environment.  In  particular,  motion 
sickness  distracts  from  the  task  since  the  discomfort  is  intrusive  and  the  gunner  cannot  readily 
concentrate.  Although  the  immediate  task  performance  may  degrade,  the  long-term  effects  are 
cumulative,  resulting  in  accumulated  fatigue  over  an  extended  period  of  time,  which  can 
adversely  affect  performance  of  subsequent  tasks  (Wickens,  Gordon,  &  Liu,  1998). 
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8.  Conclusion 


The  perceived  workload  for  the  targeting  tasks  is  a  function  of  the  skill-  and  rule-based 
components  of  reasoning,  which  are  formed  from  mental  workload  measures  of  task  attention, 
SA,  and  motion  sickness,  at  least  for  the  data  collected  in  our  field  experiment  (Smyth,  2002b). 

A  factorial  analysis  of  the  mental  workload  measures  forms  a  cognitive  component  space  that 
corresponds  to  an  SRK  model  of  information  processing  (Rasmussen,  1983).  The  measures  tend 
to  align  with  the  components,  depending  on  the  level  of  reasoning.  For  example,  the  cluster  for 
the  task  attention  loading  appears  to  be  associated  with  a  skill-based  reasoning  that  is  focused  on 
the  targeting  tasks  without  much  loading  on  the  rule  or  knowledge  components.  In  contrast,  the 
motion  sickness  and  SA  demand  measures  are  clustered  along  the  rules  component.  Finally,  the 
situation  understanding  measures  are  clustered  along  the  knowledge  component. 

The  mental  workload  measures  are  explained  by  a  micro-state  activity  model  of  the  targeting 
tasks  that  is  based  on  an  interpretation  of  human  reasoning  as  the  product  of  inter-linked 
perceptual,  cognitive,  and  psychomotor  processors  (Card,  Moran,  &  Newell,  1983).  The  model  is 
expanded  to  include  pipelined  skilled  behavior  for  task  performance  and  body  movement  to 
compensate  for  motion  sickness,  as  well  as  ruled-based  behavior  for  the  assessment  of  situation 
demand  and  motion  sickness.  Further,  the  model  includes  task  switching  and  orienting  responses 
to  audio  cues  as  intrusive  stimuli.  The  development  follows  a  generic  information-processing 
model  for  which  production  rules  are  developed.  The  results  of  the  cognitive  component  space 
analysis  are  used  to  further  refine  the  model  at  the  different  levels  of  processing.  It  is  argued  that 
the  mental  workload  measures  correspond  to  different  configurations  of  the  model  processors 
activated  at  the  skill  and  rule-based  level  for  the  tasks  of  targeting  and  monitoring  motion 
sickness.  Of  interest  is  that  the  loading  values  generated  by  the  processor  configurations 
significantly  agree  with  the  experimental  values,  following  correction  for  the  differences  in 
scaling.  Furthermore,  significant  matching  of  different  configurations  of  the  model  processors 
occurs  for  the  perceived  workload  and  the  demand  dimensions.  The  results  suggest  a  common 
micro-activity  model  source  for  these  different  facets  of  human  behavior. 

The  loading  values  generated  by  the  micro-state  model  are  used  in  an  expanded  IMPRINT  task 
analysis  workload  simulation  modeling  program  for  the  inclusion  of  SA  and  motion  sickness.  In 
this  process,  the  measures  are  scaled  on  the  7-point  (0  to  7)  loading  factors  scale  (McCracken  & 
Aldrich,  1984),  used  for  allocated  attention  in  the  program.  Note  that  in  this  simulation,  the 
definitions  of  loading,  workload,  and  total  loading  are  tied  to  the  micro-model  activity  as 
separate  concepts.  In  particular,  total  loading  is  computed  as  a  weighted  sum  of  the  loading 
values  on  the  processors,  in  which  the  weights  are  the  ratio  of  the  duration  of  the  processor 
activity  to  the  behavior  duration  for  the  total  loading,  as  determined  from  the  micro-model.  The 
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use  of  these  concepts  ensures  definite  relations  between  the  simulation  output  and  the 
experimental  measures  of  SA  and  motion  sickness,  as  well  as  perceived  workload. 

A  remaining  issue  of  research  interest  is  the  long-term  effect  of  demand  loading  on  crew 
performance.  An  increase  in  workload,  demand  on  SA,  or  the  chance  of  motion  sickness  may 
increase  the  psychological  stress  on  the  user  of  the  indirect  vision  system.  While  the  immediate 
task  performance  may  be  degraded,  the  long-term  effects  may  be  cumulative,  resulting  in 
accumulated  fatigue  over  time,  which  can  produce  errors  and  adversely  affect  performance  of 
subsequent  tasks. 

Finally,  the  task  load  performance  and  mental  workload  models  derived  in  this  report  do  not 
make  explicit  the  higher  cognitive  processes  that  are  used  in  targeting  tasks.  A  review  of  the 
human  information  processing  literature  suggests  cognitive  mechanisms  that  are  needed  for  a 
higher  level  model.  The  superposition  of  executive  cognitive  processes  by  an  expert  system, 
which  consist  of  production  rules,  is  needed  to  support  the  explicit  declaration  of  rule-based 
reasoning.  These  higher  cognitive  processes  are  used  to  develop  and  apply  the  rule-based 
schematics  that  guide  and  direct  the  skill-based  behavior.  An  essential  element  of  this 
supervisory  process  is  the  development  and  maintenance  of  a  mental  model  of  the  task  and  the 
battlefield  situation. 


9.  Recommendations  for  Further  Research 


The  author  recommends  that  the  micro-state  model  approach  developed  in  this  study  be  used  in 
the  analysis  of  performance  and  subjective  evaluation  data  collected  from  experiments.  The 
approach  provides  insight  into  the  functioning  of  the  human  at  the  information  processing  level. 

The  author  further  recommends  that 

•  the  micro-state  model  be  expanded  to  include  production  rules  and  mental  models  of 
the  tasks  and  situation.  The  model  should  also  include  training  and  motivation. 

•  higher  cognitive  functions  be  incorporated  for  multiple  tasks  involving  crew 
interaction  and  communications,  and  navigation.  The  present  study  is  limited  to  targeting  tasks 
and  does  not  consider  the  higher  cognitive  functions  that  are  required  of  future  combat  vehicle 
operators.  The  study  should  include  automated  adaptive  aiding  for  the  performance  of  multiple 
tasks  in  future  combat  systems. 
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